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ABSTRACT 



The report summarizes the results of the National Evaluation Program 
(NEP), Phase I Study of Prosecution Management Information Systems (PMIS's), 

The report Ihcludes discussions and findings regarding: the nature and 

scope of PHIS use; state-of-the-art in PMIS use; transfer potential of PMIS 
projects; funding of PMIS systems; organization of PMIS projects; the avail- 
ability of evaluation data; and the feasibility of PMIS evaluations. 

Significant findings include: 

1. Integrated systems, supported by a centralized computer facility 
serving multiple criminal justice agencies that share the data base, are the 
most common and most effective type of PMIS, 

2. Existing automated systems are being used effectively, in general, 
to support prosecutors' day-to-day case processing operations. Howevei', few 
prosecutors, with the exception of PROMTS users, are using their PMIS as a 
strategic management tool to support their policy development and decision- 
making processes. The periodic exchange of information and ideas among 
prosecutors attending PROMIS User Group meetings appears to be the reason for 
this exception. PMIS contributions to attainment of prosecutor goals have 
been less than expected by the prosecutors. This situation dlso indicates a 
need for prosecutors to exchange information about PMIS operations to either 
develop more realistic expectations or to learn about applications that will 
help their PMIS to meet their expectations. 

3. PMIS transfer has not yet proven less costly than development from 
scratch as a means of acquiring an automated system. In previous PMIS trans- 
fers, agencies have tended to underestimate the difficulty of modifying the 
incoming system to meet local requirements. Frequently, technical assistance 
should be sought from experienced data processors to effectively implement the 
PMIS. 

4. Federal funding has been a primary factor in both the transfer and 
local development of PMIS projects. Local jurisdictions have, thus far, 
assumed the financial burden of supporting PMIS operations when Federal funds 
run out. Future PMIS implementations will continue to rely heavily on^'Z^deral 
fundihg and local jurisdictions will be expected to fund future PMIS operations 
when Federal funds are .exhausted. Allocation of funds for PMIS development, 
transfer and operations should encourage the capture of benefits resulting 
from previous investments in such systems at all levels of government. 

5. Data for PMIS cost-benefit analysis and feasible methods for mea- 
suring PMIS impact on the criminal justice system are available to support 
in-depth PMIS evaluations. Independent PMIS evaluations are needed to provide 
Federal, state and local government officials the ipformation that they need 
to justify PMIS funding. 
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EXECUTIVE SUMMARY 



This National Evaluation Program (NEP), Phase 1 Study of Prosecution 
Management Information Systems, sponsored by the Law Enforcement Assistance 
Administration (LEAA), was conducted by Westat, Inc. in affiliation with the 
Center for Management and Policy Research, Incorporated (CMPR). 

For this study, a prosecution management information system (PMIS) is 
defined as the people, procedures, hardware (e.g., office and computer equip- 
ment), and software (e.g., computer programs and associated documentation) 
used by a prosecutor to create and use case information to achieve prosecution 
goals. 




The study was conducted in two segments. The objectives of the firqt 
segment were; to identify and describe the nature and extent of PMIS usage 
and the problems, costs and benefits of such use; and tg/determine tfie avail- , 
ability of data for in-depth evaluations. Information to meet these objec- 
tives was gathered by a literature search, mail and telephone surveys of more 
than 600 prosecution agencies, on-site visits to 17 operational PMIS projects 
and discussions with members of the nationwide criminal justice community. 

The second segment of the study was designed to determine the extent to 
which PMIS's can be objectively evaluated and to test evaluation methods. 

These objectives were met by performing evaluation feasibility tests using 
data collected from six PMIS projects that were selected as test sites. 

Increasing criminal caseloads combined with limited resources have moti- 
vated many prosecutors to consider automation as a means; of improving office 
operations and the management of resources. Survey data, collected during the 
- first segment of this study, indicate that f’MIS projects are being used or 
planned extensively among larger prosecution offices. About 80 percent of 
offices with 25 employees or more either use a PMIS or are planning a PMIS. 

Of those reportedly using a PMIS, 37 percent had some version of PROMIS. Of 
those reporting plans to use a PMIS, 70 percent were planning some version of 
PROMIS, indicating a trend toward the use of PROMIS. (For the purpose of this 
report, the acronym "PROMIS" refers to a group of software packages developed 
for LEAA by the Institute for Law and Social Research (INSLAW); the acronym 
"PMIS" includes both PROMIS and non-PROMIS systems.) 

Based on survey data, it is estimated that more than $30 million has been 
spent over the past 10 years on PMIS development and in excess of $20 million 
is spent each year on PMIS operations. PMIS evaluations will be useful to 
Federal officials and state planning agencies in assessing PMIS funding 
policies and allocating funds among competing projects, particularly in light 
of diminishing criminal justice budgets. Criminal justice agencies that use a 
PMIS and data processing facilities that operate them will find PMIS evalua- 
tions useful in budget negotiations because PMIS costs and benefits will be 
identified and PMIS impacts on the criminal justice system will be defined. 
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Evaluation results will also assist PMIS users and operators in determining 
changes or additions to PMIS operations to make them more effective. 

Several sets of issues were developed during the course of the study. 

The first set of issues concerns the organizational context of PMIS projects. 
Several types of PMIS projects were observed. One type is the prosecutor 
dedicated PMIS. In this type of system, a small computer is housed in the 
prosecutor's office complex and operated by members of the prosecutor's staff; 
only prosecution personnel have direct access to the system, although PMIS 
outputs (e.g. , schedules and caseload reports) may be provided to other 
agencies, such as the courts. Another type of project is where the computer 
system is housed and operated by one agency (e.g., the prosecutor), and data 
is shared with another agency (e.g., the courts or the police) via direct, 
on-line access to the PMIS. For the purpose of this study, these systems are 
said to have "limited- data sharing." 

The other types of projects observed are called, here, "integrated sys- 
tems." In these types of systems, a central data processing facility (city, 
county or state) operates a large scale computer in support of PMIS opera- 
tions, and the PMIS data base is shared by several criminal justice agencies. 
Among integrated systems, data may be shared within a jurisdiction by several 
criminal justice agencies (e.g., police, prosecutor, courts, and corrections); 
data may be shared between jurisdictions (e.g., on a regional basis); or data 
may be shared on a statewide basis. 

Integrated systems, as a group, are used by most of the jurisdictions 
surveyed; among respondents to the mail and telephone surveys, 80 percent 
indicated that their PMIS is shared extensively; of the projects included in 
the site surveys, 82 percent operate integrated systems. It is difficult to 
secure the interageincy cooperation needed to develop an effective integrated 
criminal justice information system Involving extensive da* sharing among 
criminal justice agencies. Yet, in jurisdictions where such extensive data 
sharing was observed, the projects appear to be more cost-effective than those 
projects characterized by limited data sharing. 

The second set of issues concerns PMIS development and operation. Due to 
budgetary and staffing constraints, an essential ingredient of the wide adop- 
tion of PMIS's has been LEAA funding support. LEAA policies have encouraged 
technology transfer, as evidenced by the strong interest in PROMIS with 37 
jurisdictions operating some version of PROMIS and 134 other jurisdictions 
either transferring or planning PROMIS. There is little evidence from this 
study to support the theory that PMIS transfer is less costly than development 
from scratch; in fact, respondents to the mail survey reported median develop- 
ment costs of $150,000 for 15 non-PROMIS projects and $175,000 for nine PROMIS 
projects. Several possible reasons for this are explored in the text. Exami- 
nation of the state-of-the-art in PMIS use focused on applications and the 
degree to which they are employed for management purposes. The most common 
applications of PMIS projects were directed at day-to-day operations by 
improving scheduling and logistical control through case and defendant 
tracking and caseload reporting. The use of PMIS outputs to support strategic 
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management policy development and decisionmaking was not a common practice, 
particularly in non-PROMIS sites where many prosecution managers do not 
understand the potential of the PMIS as a management tool. 

The third set gf issues concerns the impact of PMIS projects on the 
criminal justice system. Prosecutors were able to rate the relative impor- 
tance of various PMIS related goals of their offices. They ranked "improve 
scheduling and logistical control" and "allocation of staff based on prose- 
cution priorities" most highly. Some operational PMIS projects have fallen 
short of prosecutors' expectations; however, this varies with different 
applications and types of systems. In general, survey results indicate that 
prosecution management's statistical information needs are satisfied more ' 
often by PROMIS projects than by non-PROMIS projects, while the reverse holds 
for day-to-day operational support needs, such as case and defendant status 
reporting. A majority of the PMIS projects report that they generate measures 
of workload, court delay, and disposition rates. Such performance measures 
are useful in assessing PMIS impact. The collective experience in prosecution 
MIS develd^ifent and operation have advanced to the point where uniform cri- 
teria can be applied and comparable measures of costs and performance can be 
developed for groups of projects. Such an in-depth analysis has been designed 
for a proposed Phase II NEP study. 

In response to a special evaluative interest expressed by LEAA, the 
final set of issues concerns the evaluation of PROMIS projects as a group. 
Several versions of PROMIS were identified, some markedly^ different from 
others. Indeed even for particular versions, the study fdisnd that transfers 
often involved substantial software and procedural modifications. This may 
have limited the advantages of PROMIS transfer in lowering development costs 
and facilitating cross-city comparisons. Without the application of uniform 
criteria and mearurement techniques, it is difficult to compare relative costs 
of projects. Based on the limited data available from the Phase I surveys, 
little evidence was found to support the notion that such transfers lowered 
development costs. A limited scope, cross-city analysis of PROMIS projects was 
accomplished by INSLAW. However, the effort involved was time consuming and 
difficult despite the aid of a generalized report generator package, supplied 
by INSLAW. What appears to be special about the effort was the continuity of 
funding and cooperative working relationship established between the jursidic- 
tions and the contractor, rather than intrinsic attributes of the PROMIS 
data bases. The text of this report provides further evaluative information 
on the various PROMIS versions. Present LE!AA funding policies favor the use 
of PROMIS, and a new PROMIS software version, with attractive tailoring 
features, is being implemented. 

Also in response to 'LEAA concerns about how future PMIS funds should be 
allocated, the study team developed recommendations that funds be allocated on 
a priority basis; first, to provide technical assistance for PROMIS transfer, 
in order to realize maximum benefit from the heavy investment already made to 
this system; second, to jurisdictions seeking alternative approaches to 
PROMIS, because some alternatives may have more innovative and economical 
applications; and third, for the development of new PROMIS applications (as 






opposed to development of new systems), to provide even greater flexibilities 
to those PROMIS versions that already exist. Assigning priorities in this 
manner will help capture a return on investments already made in systems 
development and transfers by the Federal and local jurisdictions, and will 
allow prosecutors to gain more experience with existing applications before 
investing in new enhancements. 

Data collection experiences and testing of evaluation methodologies, 
performed during the feasibility study, have resulted in the findings that: 
following a period of 24-26 months of PMIS operations, data are available from 
PMIS projects in sufficient detail to support cost-benefit analysis; PMIS 
performance measures can be derived from records such as the PMIS computer 
files, statistical reports generated either by the PMIS or manually, and/or 
abstracted from manually kept case files; and time series analysis, employing 
factor analysis techniques and supplemented by a set of control variables 
derived from on-site interviews and observations, provide feasible though not 
infallible methods for evaluating the PMIS impact on the criminal justice 
system. 

The field investigations reported here have been necessarily limited in 
scope. They have indicated the feasibility of conducting in-depth evaluations 
of PMIS projects, but they have not constituted such in-depth evaluations. 

Also, with just six test sites, the sample is too small to do any cross- 
sectional analysis. The latter type of design, coupled with time series anal- 
ysis, offers the best opportunity for performing in-depth evaluations. Thus, 
it remains to examine the feasibility of constructing and analyzing a cross- 
jurisdictional time series data base for evaluating PROMIS and non-PROMIS 
projects. In the process of constructing and analyzing such a data base, a 
set of actudl in-depth evaluations should be carried out as an extension of 
the research methodology tested in the field investigations reported here. 

The research should include the use of non-PMIS generated data to supplement 
the analysis. 

Several classes of PMIS projects were identified during the study. 

Projects classified as integrated systems with data sharing among criminal 
„ justice agencies appear to be more cost effective than others. A cooperative 
research project involving about 30 jurisdictions in constructing a cross- 
sectioned time series data base would create a powerful vehicle for performing 
in-depth evaluative research.* It would also create the opportunity for 
exchanging user information, disseminating research results and, in general, 
educating and upgrading PMIS users. By Including both PROMIS and non-PROMIS 
users, more broadly applicable and realistic evaluations and exchanges of 
information would result. This would provide the greatest opportunity to 
assure that the PMIS is well understood and used for both operational and 
management purposes, and that the impact of such use can be measured and 
evaluated. The implications of such research for the Bureau of Justice 

I t 

. ! 

*Much of the necessary data are being compiled under the National Institute 
of Justice Cross-Jurisdictional Analysis. 
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statistics program are apparent. The participating PMIS projects could be 
uLd to generate a wide array of comparable criminal justice statistics 
support of both longitudinal and cross-sectional analysis. 

Another issue to be explored in evaluating PMIS projects is the predic- 
tion of criminal justice system costs and performance. The application of 

stochastic (probabilistic) modeling and simulation p=5s“irojLts 

exolored because such predictions are relevant in evaluating PMIS Projects, 
fomoarisons of actual with predicted results should assist in advancing the 
state of the art of evaluative and modeling research, as well as being intrin- 
fifallf useful in improving PMIS and criminal justice statistics programs. 

An evaluation of the comparative utility and of PMIS generated statistics 
versus periodic surveys by independent statistical agencies (Census or BLS 
:Sls)Tou?d be pertemL. There are inherent ® 

in developing and utilizing both approaches in an ongoing B3S sponsors 

statistical program. 







I. INTRODUCTION 



A. Purpose of Report 

This report summarizes the results of the National Evaluation Program, 
Phase I Study of Prosecution Management Information Systems, conducted for 
the Law Enforcement Assistance Administration (LEAA) by Westat, Incorporated 
of Rockville, Maryland in affiliation with the Center for Management and 
Policy Research, Incorporated of Washington, D.C. It provides a description 
of the methodology used in the study and a summary of the issues and findings. 

B . Background 

The prosecutor, in addition to prosecutorial duties, is responsible for 
management of the information that flows through his or her office. The 
information of primary concern to the prosecutor pertains to cases filed in 
court for prosecution. Case information includes data regarding; the de- 
fendant; the offense; witnesses; investigations; assignment of assistant 
prosecutors, defense attorneys and judges; schedules for hearings, trials and 
sentencing; and, final disposition of the cases. 

The people, procedures, hardware, and software used by the prosecutor to 
manage case information is called, for the purpose of this study, a Prosecu- 
tion Management Information System (PMIS). 

A PMIS may be operated by manual methods; it may be automated by em- 
ploying a general purpose computer; or, it may be semi-automated, that is, it 
may function by using a combination of manual and automated techniques. 

In a manual system, the procedures include; the origination of data by 
preparing documents in handwritten form or by use of a typewriter; the distri- 
bution of those documents to users of the information; and, storage of those 
documents in containers such as file cabinets. The hardware, then, consists 
of the office equipment (typewriters and file cabinets, for example) used to 
prepare and store the data. The manual PMIS was excluded from the scope of 
of this study. 

An automated PMIS uses ja computer to store, lanipulate, retrieve, and 
disseminate case information. Computer Capabilities provide prosecution 
personnel a quick and easy method of tracking cases and defendants; and they 
permit the automatic production of reports to support daily operations (e.g., 
schedules and dockets), management of resources (case loads for assistant 
prosecutors, for example), and statistical analysis (e.g., case disposition 
rates and case processing times) . 

Automated PMIS procedures Vary widely. The origination of data, for 
example, may be accomplished in the same manner as in a manual system with 
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an extra copy of documents being prepared as the "source document" for 
( entering data into the computer system. Data originated in this manner may 

be prepared for entry into the computer system by either "off-line" or 
"on-line" methods. Using the off-line method, source documents are normally 
delivered to a processing facility where data are prepared for entry into the 
computer by keying the data into punched cards or onto magnetic tapes, with- 
out involving the computer. On a scheduled basis, the batch of records thus 
( generated are fed into the computer for processing, hence the term "batch 

processing".* On-line data entry may also be accomplished by sending source 
documents to a central data processing facility for keying directly into the 
computer. However, the most common method of on-line data input is to use 
remote data entry devices, usually located in the office originating the 
data. 

( 

On-line operations are accomplished with terminals (keyboard devices), 
such as electric typewriters, that are connected to the computer either 
directly by a cable, or from a remote location** by telephone lines. In 
either case, the terminals provide direct access to the computer system. 

( The type of terminal employed will determine whether the computer is 

engaged for every terminal operation. Some devices, called "buffered" 
terminals, have the capability to store data within themselves and some, 
called "intelligent" terminals, have the capability to be programmed to 
perform basic functions such as data editing. "Intelligent" or "buffered" 
terminals do not engage the computer for every operation (e.g., information 
t may be stored in the terminal during data entry operations and subsequently 

fed, on-line, to the computer in a single data stream consisting of all the 
records stored in the terminal). Unbuffered terminals, on the other hand, do 
not have a storage capability and, therefore, engage the computer for each 
operation. 

C The equipment configuration for terminals also varies widely. A common 

practice is to include with the keyboard a video display device (cathode ray 
tube - a TV screen) as a component of the terminal. This device is used to 
display formats (called "screens") that aid the terminal operator in entering 
data or aid in keying in commands to the computer to perform specific pro- 
cessing transactions (e.g., to inquire about the trial date for a defendant). 

C 

* Batch processing, aa. used in this study, also refers to the processing 
that is accomplished at given intervals (e.g., daily) whereby data that 
have been entered into the PMIS during a specific period of time (preceding 
24 hours, for example) have been sorted into groups (batches) before the 
^ processing of scheduled reports begins. Batch processing usually involves 

the manipulation of many records, in comparison with on-line processing 
which may involve retrieving only one or a few records; and, batch proces- 
sing is usually accomplished during a time period when on-line operations 
are minimal. 

**Remote location, as used here, means that a terminal is too far from the 
^ computer to be connected by cable; it may be located in the same building, 

an adjacent building, or miles from the computer fability. 
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Another terminal configuration may include an on-line printer. This config- 
uration permits the rapid generation of printed reports or documents in 
response to a request by the terminal operator. 

Use of an on-line printing capability allows variations to data entry 
procedures. As an example, rather than preparing source documents for use in 
data entry operations, as previously discussed, certain data or commands can 
be transmitted, on-line, to the computer from a remote terminal. The so- 
called source document can then be generated by the PMIS via the on-line 
printer (e.g., arrest reports using on-line booking operations, or generation 
of subpoenas, as performed by Norfolk's TRACER system). 

Various documents (such as arrest reports, prosecution work sheets, 
schedules, dockets, and disposition reports) are needed in hard copy form to 
satisfy prosecutor and court operating procedures and to comply with estab- 
lished recordkeeping requirements. In a non-automated system, all of these 
documents are prepared by manual methods, whereas in an automated system the 
extent of PMIS generation of such documents will depend on: what data ele- 

ments are entered into the system; and, what computer programs have been 
written to satisfy PMIS processing and output requirements. 

Computer programs are instructions, usually written in a standard pro- 
gramming language such as the Common Business Oriented Language (COBOL), that 
tell the computer how to handle and store input data, how to manipulate the 
data to satisfy processing requirements, and what outputs are to be generated. 
These programs, along with other documentation that describes system proce- 
dures, constitute the software for an automated PMIS. 

The data elements that may be entered into the PMIS consist of items 
such as: the defendant's name, age, sex, race; offense charged; date of 
offense; witnesses names, addresses, phone numbers; dates of hearings, trial 
and sentencing; and so forth. The number of different data elements, relevant 
to a specific defendant or case, entered into a PMIS depends largely on which 
criminal justice agencies are "users" of the PMIS. 

A PMIS user, for the purpose of this study, is an agency that has direct 
access to the PMIS and is authori2;ed to retrieve and/or update PMIS defendant 
or case records. Users, then, "share" the PMIS data. 

The extent of data sharing is a major PMIS characteristic and influences 
the number of different data elements stored in the system. A PMiS may 
support only the prosecutor. In this type of system only data of concern to 
the prosecutor are entered into the PMIS; items of interest to other agencies, 
such as the police (e.g., identification of stolen property) or the court 
(e.g., accounting for the collection of fines), would be excluded from the 
system. Another type of PMIS is one shared by the prosecutor and courts. 

The extent of data sharing is rather limited in this type of system, but the 
data elements would include items of interest to both the prosecutor and 
courts. In other words, the greater the number of system users, the greater 
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the number of data elements in the data base. An automated system, in fact, 
may include several or all criminal justice agencies within a jurisdiction. 

This type of system, called an "integrated system" in this study, would 
have specific files, within the data base, devoted to each user. Users, 
then, would update (add or change records) their own files; with appropriate 
interagency ageements, they could retrieve information from other user files 
(or portions of the other files); but, in general, they would not be able to 
update the files devoted to other users. 

The capability to "talk" to computers via telecommunication facilities 
allows data sharing by more than one jurisdiction, and, if desired, can be 
accomplished on a statewide, national, or worldwide basis. 

The capability to talk to computers is not limited to man-machine communi- 
cations; computers can talk to one another (machine-machine communications). 
This type of arrangement, called a computer "network", is accomplished through 
telecommunications facilities and by the use of computer programs written 
specifically to control the network operations. 

Computer networks provide processing capabilities that are, obviously, 
beyond the realm of single computer operations. In the criminal justice 
community, for example, individual jurisdictions could have their own com- 
puters linked to a state level computer system. On a prearranged schedule 
(e.g., daily), the computer at each jurisdiction could automatically transmit 
selected data (e.g., warrants) about local offenders. If an offender from 
one jurisdiction is apprehended in another jurisdiction, the apprehending 
jurisdiction could make an inquiry to its local computer to determine the 
status of the offender. If the offender is not wanted in the apprehending 
jurisdiction, the local computer file will not contain the warrant informa- 
tion. The inquiry would then be automatically transmitted to the state 
system; the warrant information would be retrieved from the state system and 
automatically transmitted back to the apprehending jurisdiction; and the 
jurisdiction that issued the warrant could be automatically notified of the 
apprehension via the computer network. 

A computer network, such as described above, could also be used to 
process periodic reports at the state level, using the data transmitted 
regularly by individual jurisdictions. This arrangement would relieve the 
individual jurisdictions of those reporting requirements. An expansion of 
this approach could be the use of the state level system (or some centralized 
system) to perform the processing necessary to generate most of the hard copy 
documents required by individual jurisdictions, even those needed on a daily 
basis. This approach would reduce processing requirements within the indi- 
vidual jurisdictions, thus reducing the size requirement for computers at 
those locations. 

Many makes and models of computers are available to support PMIS opera- 
tions. The predominant characteristic, however, is the computer size. A 
PMIS designed to support only the prosecutor, for example, may employ a 




minicomputer or medium scale computer that could be housed and operated in 
the prosecutor's office complex. Another approach to a prosecutor dedicated 
system could be the use of a large scale computer located in and operated by a 
centralized city, county, or state data processing facility. In the latter 
situation, the computer system would have files devoted to the PMIS and 
accessible only by the prosecutor. 

A centralized, large scale computer would also be used to support an 
integrated system or a system designed to support multiple jurisdictions. In 
a computer network, as previously discussed, individual jurisdictions could 
be equipped with minicomputers that would be linked to a large scale computer 
at a central (state or regional) facility. 

Several characteristics can be used to describe the difference between, 
mini, medium and large scale computers. The most distinguishing character- 
xstics concern physical size, costs, main memory capacities* and personnel 
resources required to operate the different systems. 1 (See Appendix A 
for references.**) 

I^^nico mputers are small in size and relatively inexpensive. Some are 
about the size of a desk and a few models (microcomputers) are small enough to 
place on top of a desk. In a typical minicomputer installation, two or three 
terminals may be used for data input and system control; one tape drive or one 
disk drive may be used for secondary data storage; and a printer would be 
connected to the system to provide hard copy outputs. The main memory unit 
may contain from 4,000 to 512,000 storage locations. However, memory capac- 
ities are being continuously increased without increasing the physical size of 
the memory unit, through the advancement of miniaturization techniques. The 
cost of purchasing a minicomputer may range from $50,000 to $200,000 or one 
may be rented for $1,250 to $5,000 a month. Such a system could be operated 
by as few as one or two people. 



* Computer main memories (often called the "core") consist of electrical cir- 
cuits within the central processing unit. The "on" or "off" condition of a 
circuit is used to represent the 0 (off) and 1 (on) digits in the binary 
numbering system. In computer jargon, a binary representation (the 1 or 0) 

IS called a "bit". Combinations of bits (usually seven or eight) are used to 
code characters for storage in the computer memory (e.g., in one seven bit 
coding scheme, 1000001 = A). The circuits used to code a character (seven 
circuits in this example) are referred to as a storage "location". Memory 
capacities, as discussed here, are measured by the number of storage loca- 
tions contained in the core. 

**Definitions of different size computers are taken from publications printed 
in 1977-1978. Costs included in these definitions may differ somewhat in 
today’s market because of inflation, but the relativity of these costs 
between computers remain the same and they relate to the costs encountered 
during the surveys conducted for this study. 
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Medium scale computers normally have the capability to handle more 
peripheral devices than minicomputers; their main memories have greater capac- 
ities; and their physical space requirements are somewhat geater than for the 
mini's. These computers may range in price from $200,000 to $1,000,000, or 
they may be rented for between $5,000 and $25,000 a month. Usually, four or 
five people, including a couple of programmers, are employed to staff a 
medium scale computer operation. 

Large scale computer facilities are typically laden with equipment and 
people. It is not unusual to find two large scale computers in one such 
facility with over 50 people employed to operate the equipment and to write 
computer programs. The extensive capabilities of these computers permit the 
use of many peripheral devices; the use of remote terminals, which may number 
over 100, requires special telecommunications equipment (modem and monitors). 
Thus, a large area is required to house a large scale computer and its periph- 
eral equipment, and separate offics' space is needed for the programming staff. 
Typical memory capacities range from 64,000 to 4,000,000 locations - some of 
the biggest systems have 30 billion locations. The costs of purchasing or 
renting large scale computers ranges widely, of course, depending on memory 
capacities and the equipment configuration. Some systems may be purchased for 
about $500,000 while others may range up to $3,000,000. Rentals may start 
around $12,500 a month and range to $50,000 or more. 

Hardware for an automated PMIS, then, consists of standard office equip- 
ment plus the equipment that makes up the computer configuration, typically: 

■ A central processing unit (CPU) - the computer main frame - 
contains the main memory (storage) unit; 

■ Secondary storage (tape and/or disk drives); 

• Input/output devices (terminals and printers, for example); 

• Telecommunications monitors and modems (where remote terminals 
are used) ; and 

• Data preparation devices (keypunch or key to tape machines - 
where off-line data entry is employed). 

There is also a correlation between the size of a computer and the spaed 
at which it operates. Large scale computers operate at speeds measured in nano- 
seconds (one-billionth of a second) while medium scale and minicomputer speeds 
are usually li.easured in microseconds (thousands of a second). 

Computer speed is an important factor when selecting a system to effi- 
ciently process very large volumes of data. Some organizations, however, may 
have a data volume that is insignificant as far as computer speed is concerned; 
it may not matter to a prosecutor, for example, whether a computer operates in 
nanoseconds or microseconds; what will matter is that a computer can effi- 
ciently process a volume of data that has surpassed the capability of the 
prosecution staff to handle efficiently using manual methods. 
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Increased caseloads, resulting in increased data volumes, and the desire 
to improve the efficiency of case processing has prompted prosecutors across 
the nation to consider automation of their prosecution management information 
system. 

A variety of automated PMIS's have been developed and implemented over 
the past 10 years to assist prosecutors in their daily operations and in 
management of resources. Of these, the largest single program has been the 
Prosecutor's Management Information Systern (PROMIS).* 

PROMTS was developed by the founders of the Institute for Law and Social 
Research (INSLAW) for the U.S. Attorney for the District of Columbia in 1971. 
The original PROMIS was a batch processing system (no on-line operations) 
designed to run on a large scale computer. Subsequently, with LEAA funding, 
PROMIS was redesigned as a transferable system and implemented in other juris- 
dictions, such as New Orleans and Indianapolis. In Milwaukee, Wisconsin, 
substantial modifications** were made to PROMIS, and the modified system was 
then transferred to other jurisdictions, such as Louisville, Kentucky. 
Improvements resulting from redesign and modification included the addition of 
on-line capabilities and the development of additional software designed to 
operate on minicomputers. 

Two minicomputer versions of PROMIS software have been developed; one, 
called MINI-PROMIS, was designed to operate with unbuffered terminals; the 
other system, called MAXI-PROMIS or new-PROMIS, has been designed to use 
buffered terminals. The two minicomputer versions of PROMIS can be run on 
large scale computers as well as on minicomputers, and both software packages 
feature a "tailoring" capability. 

The PROMIS tailoring capability^ allows individual jurisdictions to 
design their own: data base (fiT§j$ and records); data entry screen and edit 

criteria; inquiry displays; indexed; and formats for output reports. This 
tailoring is accomplished by use of an on-line, interactive computer program 
(using a question and answer format) that takes the user, step-by-step, 
through a series of tables that define the files, transactions, data element 
characteristics, and so forth, as specified by the individual jurisdiction. 

MINI- and MAXI-PROMIS programs have been written in the COBOL programming 
language and have been designed to operate on a number of different Computers. 

* For the purpose of this report, the acronym "PROMIS" refers to the system 
developed for LEAA by INSLAW; the acronym "PMIS" refers to both PROMIS and 
non-PROMIS systems. 

**For the purpose of this report, modified versions of PROMIS, such as Mil- 
waukee's 3USTIS, are called Hybrid versions of PROMIS. This designation 
reflects inclusion of Milwaukee as an "operational" PROMIS project in 
INSLAW publications, while LEAA officials and Milwaukee staff consider 
JUSTIS as a non-PROMIS project. 
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These features, plus the tailoring capability, are intended to give the pros- ! 

eoutor maximum flexibility in designing system software and in the selection 
of hardware. 

In addition to the batch, on-line, hybrid, MINI-PROMIS and MAXI-PROMIS 
versions, manual and semiautomated versions of PROMIS also exist. As of 
January 1980, INSLAW reported that; 37 jurisdictions are operating some 
version of PROMIS; 71 other jurisdictions are in the process of transferring 
PROMIS; and 63 other jurisdictions are planning for or evaluating PROMIS.^ 

One might expect that a system developed primarily by one organization, 
such as INSLAW's PROMIS, would be similarly configured in many of its opera- 
tional sites. In fact, PROMIS project configurations vary widely. In three 
PROMIS sites (Los Angeles, New Orleans and the District of Columbia, for 
example), data entry is accomplished in a batch processing mode with on-line 
inquiry capabilities; two of those systems run on large scale, centrally 
located computers and share their data base with other users; the New Orleans 
system, however, is run on a large minicomputer that is operated and used only 
by the prosecutor. At other PROMIS sites (Louisville and Milwaukee, for 
example), large scale, county operated computers function in an on-line mode 
and the system is shared extensively by other users. Large scale PROMIS 
operations, such as Tallahassee, Florida, provide on-line support to multiple 
jurisdictions; and MAXI-PROMIS is being implemented in Colorado using a large 
scale computer tc provide support on a regional basis. In Portland, Oregon, 
on the other hand, MAXI-PROMIS is being implemented on a large scale, county 
operated computer, but will be used only by the prosecutor. 

Non-PROMIS projects also vary widely, insofar as system configuration 
and extent of data sharing are' concerned. In Oakland, California (the DALITE 
system) and Boston, Massachusetts (the Case Management System), for example, 
minicomputers operated by prosecution personnel are dedicated to functions 
of the prosecutor's office. A more common situation is a large scale computer, 
operated by a central city or county data processing facility, providing on- 
line support to the prosecutor and other system users. The extent of data 
sharing will vary among these centralized systems. In Baltimore, Maryland, 
for example, the system supports the courts and the prosecutor; other systems, 
such as those located in Miami, Florida, and in Fort Worth, Texas, support all 
criminal justice agencies throughout Dade and Tarrant countries, respectively; 
and, the PMIS in Norfolk, Virginia, supports criminal justice agencies within 
the Tidewater Region. 

The system used in Norfolk can also be used as an example of a computer 
network: the PMIS, called TRACER, interfaces with the Tidewater Electronic 

Network of Police Information (TENPIN) which, in turn, interfaces CPU to CPU 
with the Virginia Criminal Information Network (VCIN) and, in turn, the 
National Crime Information Center (NCIC). 

Another variation to large scale PMIS operations can be seen in Oklahoma 
City, Oklahoma where a county data processing facility provides support on a Is 

statewide basis: five counties operate terminals connected on-line to the j • 



computer for data input and for query/response transactions; the other counties 
send source documents to the Statistical Analysis Center (SAC) of the Oklahoma 
Crime Commission for input to the computer and are able to obtain responses to 
inquiries by telephone contact with SAC where the inquiry is made via on-line 
terminal and the response received by SAC is relayed verbally using the 
telephone. 

Implementation of non-PROMIS systems has, for the most part, been based 
on independent development by individual jurisdictions; transfer of non-PROMIS 
systems is not a common practice.* Development of non-PROMIS systems and 
transfer of PROMIS have been accomplished by in-house personnel, contractors, 
or a combination of the two, depending on the availability of in-house person- 
nel with the appropriate expertise. 



C. Need for PMIS Evaluation 

Increasing criminal caseloads coupled with limited resources have moti- 
vated many prosecutors to consider automation as a means of improving manage- 
ment and operations. Indeed, over 175 of the larger prosecution offices 
surveyed by mail during this study (having more than 25 employees) , either 
have an operating PMIS or are in the process of planning such a system. 

Based on the survey data collected during this study, it is estimated 
that in excess of $30 million has been spent in the last decade developing 
A various types of automated systems to serve prosecutors and over $20 million 

is spent each year on the operation of such systems. Until quite recently, 
the outlook was for substantial expenditures to continue due to increasing 
interest of prosecutors in automation, the current funding policies of LEAA, 
and interest in criminal justice research. In light of diminishing criminal 
justice budgets, it is expected that the results of an evaluation of PMIS's 
O will be useful to Federal officials and state planning agencies in assessing 

PMIS funding policies, reviewing grant applications, and allocating funds 
among competing projects. Prosecutors, courts, other criminal justice agen- 
cies, and state and local government data processing service organizations 
should find evaluation measurements useful in identifying elements of PMIS's, 
and their associated implementation projects, that have been successful and 
O effective in improving prosecution and court performance. Prosecutors, 

judges, and prosecution/court admini/strators will gain insights into factors 



*The LEAA 1976 Directory of Automated Criminal Justice Information Systems 
lists 24 systems with the function of "prosecution management"; of the 20 
non-PROMIS systems on this list, none are transferred systems. The Criminal 
Oriented Records Production Unified System (CORPUS), operated by Alameda 
County, California, was originally transferred from Santa Clara County, 
California, but extensive modifications have been made to the original 
system; CORPUS is the only transferred non-PROMIS system that was examined 
during this study. 
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contributing to PMIS success and failure, methods of measuring performance, 
and approaches to improving PMIS projects and prosecution/court performance. 
Information resulting from evaluations should also assist in determining 
changes or additions to make PMIS projects more effective. 

D. Purpose and Scope of the Study 

This NEP Phase I study was conducted in two segments. The first segment, 
performed during the period of October 1978 through November 1979, was 
designed to identify and describe the nature and extent of prosecution MIS 
usage and the problems, costs and benefits of such use. This segment was also 
designed to determine if data are available to support in-depth, objective 
evaluations (NEP Phase II studies). Information needed to meet the study 
objectives of this first segment was gathered by means of a mail survey of 594 
prosecutorial agencies and by site visits to 17 locations where automated 
information systems used by prosecutors are currently in operation. Mail 
survey and site visit data were augmented by information obtained from avail- 
able literature (see Appendix B) and from telephone contacts with individuals 
throughout the judicial system. 

The second segment, initiated in January 1980 and concluded in June 1980, 
was designed to determine the extent to which prosecution management informa- 
tion systems can be objectively evaluated and to test evaluation methodologies. 
Site surveys of six PMIS projects were conducted for the purpose of satisfying 
the objectives of this study segment. These surveys included four PMIS 
projects that had been visited during the first- segment of the study plus two 
other sites that were in the process of implementing the latest version of 
PROMIS (MAXI-PROMIS). 

Appendix C contains a list of personnel interviewed during site visits 
in both segments of the study. 



E. Documentation of the Study // 

/J 

This study has been documented in a r/yort entitled "National Evaluation 
Program Phase I Final Report, Prosecution/ Management Information Systems," 
which consists of three volumes; ) / 



Volume I - Evaluability of Systi^ms - defines the objectives and 
scope of the study; outlines theWaluation framework used; describes 
various types of prosecution mank^ement information systems that were 
observed; discusses the state-ofA ne-art and state of knowledge about 
these systems^ and presents the fi\^jdings from the first segment of 
the study regarding quantitative results, surveys of PROMIS projects, 
and the investigator's judgmental assessments of PMIS projects. 



Volume II - Feasibility Tests - covers the second segment of the 
study and defines the purpose of the tests and data collected; 
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i outlines the evaluation design; identifies the test sites and 

! ’ describes the criteria used for their selection; and discusses the 

test results. 

! ■ Volume III - Site Visit Reports - contains a case study of each 

I project surveyed during the first and second segments of the Phase I 

study. Each report describes the characteristics of the organiza- 
' tion and PMIS surveyed, describes the judicial process within the 

jurisdiction, and discusses the findings, observations and investi- 
I gator's judgmental assessments regarding the PMIS. 

j 

I 
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II. EVALUABILITY OF SYSTEMS 



A. Introduction 

Since the characteristics and environments of PMIS projects vary widely, 
the first segment of this study was designed to survey these projects and 
develop comparable information within a structured evaluation framework. A 
major objective of this segment was to assess the evaluability of the various 
PMIS projects. 

B . Methodology 

Two primary techniques were employed in gathering information about what 
exists: (1) a mail survey with telephone followup, and (2) a set of site 

visits . 

1 . Mail Survey 

The sample for the mail survey was selected from a Bureau of Census 
tape file which contained a total of 8,715 agencies.* Excluded from the 
sampling process were agencies with less than five employees and agencies 
which! 

• Provide legal services onlyj 

1' C’ 

• Deal exclusively with juvenile cases; 

• Deal exclusively with traffic violations; or 

, ■ '' 

• Are branch offices. 

After eliminating out-pf-scope agencies that could be identified 
by data on the tape, the remaining 1,533 agencies Were stratified by number 
of employees. Ail agencies in stratum A (25 or more employees) were 
included in the sample. Agencies in stratum B (10-24 employees) and C (5 -r 
9 employees) Were sequenced by level of jurisdiction (state, county and 
municipal) and by census division; choosing two sites at random out of each 
block of four sites on the list, half of stratum B was selected; and simi- 
larly every fourth agency in stratum C was selected. As shown in Table 1, 
the result was a sample of 699 agencies from which 105 were deleted as out- 
of-scope. The remaining 594 in-scope agencies formed the basis for the 
mail survey. 



*State and Local.^Prosecution and Civil Attorney Systems, U. S. dtepartment of 
Justice, March, ^978. 
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Table 1 . Mail Survey Sample 



2. Site Surveys 

The following criteria were used to select projects for site surveys: 

■ Automated system operational for at least three 
months; 

• Staff is cooperative; 

• Documentation and data are available; 

• Applications are of general interest; 

• Availability of data for performance measures; 

• Project is innovative and widely applicable; 



• There is an interesting organizational aspect, such as 
regional time-sharing or an integrated police/prosecutor/ 
court/corrections interface; aqd/or 



• A significant number of prosecutor MIS applications are 
represented. 



Based on a review of available literature, 50 PMIS projects were 
selected as prime candidates for on-site surveys. As a result of telephone 
contacts with site personnel and responses to mail surveys, a final selection 
of 17 locations for site visits was made, A list of these sites, and a 
description of the type of PMIS used is shown in Exhibit 1. 
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Exhibit 1 . Projects Selected for Site Survey 



PROJECT NAME (ACRONYM) 

AND LOCATION (COUNTY) 

1. Prosecutor's Management 

Information System (PROMIS) 
Los Angeles (Los Angeles) CA 



Criminal Oriented Records 
Production Unified System (CORPUS) 
and District Attorney Automated 
Legal Information System (DALITE) 
Oakland (Alameda) CA 



Criminal Justice Information 
Center (CJIC) 

San Jose (Santa Clara) CA 



Superior Court/County Clerk 
Information System (SUPER/CC) 
Santa Ana (Orange) CA 



Automated Court Information 
System (ACIS) 

San Bernardino (San Bernarcjino) CA 



Dade County Criminal Justice 
Information System (CJIS) 
Miami (Dade) FL 



7. Judicial Automated Records 
System (JARS) 

Waukegan (Lake) IL 



TYPE OF SYSTEM 

Integrated system 
Large scale computer 
Operated by county 
Limited data sharing 

CORPUS - Integrated system 

Large scale computer 
Operated by county 
Extensive data sharing 

DALITE - Prosecutor dedicated 
system 

Minicomputer 
Operated by prosecutor 
Access by prosecutor 
only 

Integrated system 
Large'scale computer 
Operated by county 
Extensive data sharing 

Integrated system 
Large scale computer 
Operated by county 
Extensive data sharing 

Integrated system 
Large scale computer 
Operated by county 
Extensive data sharing 

Integrated system 
Large scale computer 
Operated by county 
Extensive data sharing 

Integrated system 
Large scale computer 
Operated by county 
Extensive data sharing “ 
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Exhibit 1. Projects Selected 


for Site Survey (Continued) 






■ 


PROJECT NAME (ACRONYM) 
AND LOCATION (COUNTY) 


TYPE OF SYSTEM 




p 




8. 


Prosecutor's Management 
Information System (PROMIS) 
Indianapolis (Marion) IN 


Prosecutor dedicated system 
Minicomputer 
Operated by county 
Access by prosecutor only 




, 


0 


9. 


Commonwealth Attorney's Tracking 
and Case History System (CATCH) - 
PROMIS 

Louisville (Jefferson) KY 


Integrated system 
Large scale computer 
Operated by county 
Limited data sharing 






(, 3 


10. 


District Attorney's Record 
Tracking Systems (DARTS) - PROMIS 
New Orleans (Orleans Parish) LA 


Prosecutor dedicated system 
Minicomputer (large) 
Operated by prosecutor 
Access by prosecutor only 






f 3 


11. 


Case Management System (CMS) 
Boston (Suffolk) MA 


Prosecutor dedicated system 
Minicomputer 
Operated by prosecutor 
Access by prosecutor only 




' 

’ ■ 




12. 


Arrest Disposition Reporting 
System (ADRS) 

Oklahoma City - Statewide OK 


Integrated system (statewide) 
Large scale computer 
Operated by county 
Extensive data sharing 






O 


13. 


Tarrant County Criminal Justice ’ 
Information System (TCCJIS) 

Fort Worth (Tarrant) TX 


Integrated system 
Large scale computer 
Operated by county 
Extensive data sharing 


1 ' , 




(:) 


14. 


Total Recall Adult Criminal 
Element Record (TRACER) 
Norfolk - Regional VA 


Integrated system (regional) 
Large scale computer 
Operated by city 
Extensive data sharing 




- It • ■ 


O 

o 


15. 


Justice Information System,, 
(JUSTIS) - PROMIS 
Milwaukee (Milwaukee) WI 


Integrated system 
Large scale computer 
Operated by county 
’ Extensive data sharing 






0 
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Exhibit 1. Projects Selected for Site Survey (Continued) 



PROJECT NAME (ACRONYM) 

AND LOCATION (COUNTY) 

16. Prosecutor's Management 

Information System (PROMiC) 
Washington, D.C. 



17. Maryland Court System (MCS) 
Baltimore MD 



C 



TYPE OF SYSTEM 

Integrated system 
Large scale computer 
Operated by city 
Limited data sharing 

Integrated system 
Large scale computer 
Operated by state 
Extensive data sharing 





C. Evaluation Framework 

PMIS evaluation is based on the notion that a prosecutor has defined, at 
least implicitly, goals for the operation of his or her office. These goals 
are normally defined in such terms as conviction rates, evenhandedness, delay, 
or other attributes of the criminal justice system.^ The PMIS is developed 
and operated in the hope that it will assist in the achievement of goals, i.e., 
that it will have a positive impact. A logically structured impact evaluation 
must be carried out within a methodological framework that: 

• Provides a general theory linking PMIS operation to impact on the 
criminal justice system. 

s Defines prosecutor goals in terms of specific element^ of PMIS 
impact. 

• Defines indicators of success — operational measures of these 
elements that can be used to quantify the extent to which PMIS 
operation has impacts that advance the prosecutor's goals. 

Figure 1 summarizes a theory of how PMIS development and operation are 
linked to impact on the criminal justice system. Ideally, development of a 
PMIS takes place in four steps: requirements analysis, system design, system 

software development or transfer, and system implementation. Once the PMIS is 
operational, it may produce three categories of outputs: operational outputs , 

such as lists of witnesses, charging instruments, or subpoenas; records of 
operations and decisions such as in individual cases, attorney assignments, 
hearing dates or motions granted; and statistical displays in which these case- 
level operations are aggregated into such measures as dismissal rates, 
average delay, or case mix. 

In turn, these outputs may modify the prosecution or judicial process in 
a variety of ways. For example, witness lists, including accurate addressses 
and/or telephone numbers, may be used to administer the "on-call" feature of a 
witness assistance program to avoid unnecessary trips to the courthouse, only 
to learn that a hearing has been continued. Up-to-date and accurate records 
of attorney assignments and hearing dates may be used to reduce the incidence 
of schedule conflicts for individual assistant prosecutors. Statistical ! 

displays of case duration may be used to monitor compliance with a speedy 
trial law. i 

Such modifications of the process as these comprise the PMIS impact. In i 

addition, by monitoring various indicators over time, the PMIS ideally pro- I 

vides feedback not only on its own impact, but on the impact of external j ' 

changes in the criminal justice process, ! 

/' ! " 

With the background provided by this overview of the theoretical linkage 
between PMIS operation and criminal justice system impact, it is possible to / 

define more specifically the elements of PMIS impacts on prosecution and to 
relate them to goals of prosecution. Figure 2 identifies two general goals 
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Figure 1 . Assumptions Linking Prosecution MIS to Impact on Prosecution/Court Criminal Justice System 
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PROSECUTION MANAGEMENT & 
ADMINISTRATIVE GOALS 


Utilize PMIS to 
more effectively 
manage & operate 
the prosecutor's 
office. (9) 


Manage the PMIS 
project effec- 
tively (10) 






PROSECUTOR OPERATIONS 
Goals 



to 

O 



[Improve Schedul- 
ing & Logisti- 
cal Control 

(X) 



Maintain Ef- 
fective Con- 
Jvictlon Rate 
( 2 ) 



Reduce avoid' 
able prose- 
cutor-re- 
lated post- 
ponements 

Improve com- 
munication 
with wit- 
nesses and 
court 

Monitor 
timeliness 
of disposi- 
tions 



Maintain 
lEvenhandedness 
(3) 



• Reduce In-j 
efficiency] 
related 
dismissals! 

& nol 
pros 

• Control 
plea nego-| 
tlations 

• Vigorously] 
prosecute 
cases witli] 
appropriate 



Maintain 
uniform 
charging 
and dis- 
cretionary! 
practices 

Monitor 
prosecu- 
tor's de- 
cision 
patterns 



Use Resources 
Efficiently 
(4) 



Monitor at- 
torney as- 
signments 
and uti- 
lization 

Streamline 
administra- 
tive prac- 
tices 



FRAMEWORK 

ELEMENTS 
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Information-Based Prosecutor Actions 
Necessary to Achieve Goals 
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PROSECUTORS MANAGEMENT INFORMATION SYSTEM (PMIS) 
Goals 


Improve Information 
System Outputs 
(5) 


Improve Data 
Handling 
(6) 


PMIS Cost Control 
(7) 


Enable Research/Evaluation 
Capability 
(8) 


• Query responses 

• Reports 

• Working 
documents 


• Collection and 
verification 

• Processing 

• Maintenance and 
update 


• Design 

• Acquisition 

• Operation 


• Offender tracking across 
cases and agencies 

« Maintenance of historical 
data files with longitud- 
inal tracking capabil- 
ities 

• Transaction based case 
tracking with many 
descriptive variables 



Indicators of Success 



ASSUMPTIONS 



INDICATORS OF 
ACHIEVEMENT 



PHIS Project Actions to: 

a Meet PHIS Objectives and 
• Support Prosecutor Actions 



Indicators of Success 



Fiq-ure 2. Evaluation Framework 
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assumed to be important to the prosecutor implementing a PMIS: to manage 

the PMIS project itself effectively, and to use the PMIS to improve office 
management and operation. The former general objective is broken down 
into four more concrete prosecution goals: improving information system 

outputs, improving data handling, controlling PMIS cost, and developing a ^ 
research and evaluation capability. The latter objective is broken down iri.o 
four operational goals: improving scheduling and control, maintaining an 

effective conviction rate, maintaining evenhandedness, and using resources 
efficiently. 

Assuming that the pi^bsecutor takes actions to manage the PMIS project 
effectively and to use P^IS outputs to satisfy his management goals, the tt.ird 
function of an evaluation) framework is to define indicators of PMIS success. 

In other words, the evalwator must define operational measures that can be 
used to quantify the extjint to which the PMIS advances the prosecution goals 
identified in Figure 2. These indicators are defined in Section II, Volume I 
of the Final Report. 

D. Analysis of PMIS Usage 

Three size groupings of prosecution offices were surveyed by mail in 
August and September 1979. A comprehensive telephone survey was conducted in 
October 1979 to tabulate a minimal set of characteristics of the non-responsive 
prosecution offices. From this telephone survey, it was determined that 
certain offices in the sample were actually out-of-seope, according to the 
survey criteria. It is apparent that the smaller the office, the less likely 
it was to respond, possibly due to a lack of staff time or a lack of interest 
in the subject matter. 

The response rates are shown in Table 2. To what extent are any statis- 
tics developed from this set of responses indicative of the universe of prose- 
cution offices in the nation? First of all, the data should be analyzed by 
size group due to the differences in rate of response as well as expected 
need for an MIS. Second, researchers should be aware of any systematic bias 
that might exist in the responding population. Based on a comparison of the 
characteristics of the non-responsive group to those of the responding group, _ 
there does not appear to be an appreciable non-response bias in the propor- 
tions who have or plan to have" an MIS, nor between PROMIS and non-PROMIS 
sites. 



The statistical tables that follow, then, should be understood for what 
they are. They present evidence that may be considered to support or refute 
certain hypotheses; they indicate patterns of PMIS use, problems, costs, and 
benefits, and comparable characteristics. They do not purport to constitute 
a representative sample of the population of prosecutors' offices, sufficient 
to make statistical inferences concerning the universe at large. Given the 
limitations inherent in that disclaimer, the data have been analyzed in 
various formats to yield insights on issdpB relevant to this study. 
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Information gathered during the 17 PMIS site surveys substantiate the 
fact that PMIS environments and characteristics differ in each jurisdiction. 

A detailed description of each surveyed PMIS project is contained in Volume 
III of the Final Report. 

The analysis and evaluation of individual PMIS support functions is an 
NEP Phase II task. PMIS support, for the purpose of this Phase I effort, has 
been synthesized to illustrate PMIS relationship to the judicial process, to 
compare prosecutor controlled systems with integrated systems, and to describe 
the types of output reports that support operational and managerial functions. 

The first analysis to be presented will indicate the nature and scope of 
PMIS use throughout the nation, based on data from the mail and telephone sur- 
veys. The second analysis addresses the nature of PMIS support , based on the 
combination of PMIS functions observed during the site surveys. 
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Table 2. Adjusted Response Rates 




*Based on in-scope agencies. 

Source: PMIS Assessment Mail/Telephone Survey, Westat, Inc., August/September 

1979 



1 . Nature and Scope of PMIS Use 




The first set of findings from the mail survey describe the extent 
to which prosecution MIS's are being used or planned. 




As Table 3 shows, MIS's are used predominately by the larger offices. 
It is interesting to note that the proportion of MIS's planned in the smaller 
offices is substantially greater than the proportion used, which may indicate 
the effect of the trend toward smaller and less expensive computers. 







The extent to which automation has been adopted by the nation's 
f larger prosecution offices is impressive. Of 109 prosecution offices with 25 

employees or more, approximately 80 percent either use or plan to use an MIS. 
On the other hand, only about 19 percent of the smaller offices either use or 
plan to use an MIS. 

Comparisons of jurisdictions using an MIS with those planning an MIS 
( reveal a substantially higher proportion plan to use PROMIS than currently 

use PROMIS. Of 43 jurisdictions that reported having a prosecution MIS, 37 
percent had some version of PROMIS; of 63 reported planning a prosecutor MIS, 
70 percent were planning to install some version of PROMIS. This may reflect 
the trend in LEAA funding policy or the effect of a recently introduced 
minicomputer version of PROMIS. These possibilities are examined later in 
( this report. 

In order to obtain insights to the nature of the various prosecution 
MIS projects, a question was asked concerning the types of systems that 
prosecutors use; or plan to use. Of 43 responding prosecution offices having 
an MIS, about 23 percent report using a minicomputer; while of 60 offices 
f planning an MIS, about 40 percent report planning for a minicomputer. This 

indicates a trend toward the use of minicomputers. 

Table 3. Proportion of Prosecutors' Offices Using 
or Planning an MIS (PROMIS/Non-PROMIS) 

(percentages in parentheses) 

C' 
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Source; PMIS Assessment Mail Survey, Westat, Inc. July, 1979 






Since the number of prosecution offices, with less than 25 employees, 
that have or plan to have an MIS is too small to analyze further, the remain- 
der of this analysis will focus on the larger prosecution offices, i.e., those 
with 25 employees or more. 

^ question was asked concerning the jurisdictional area served by 
the prosecutor. The results show that about three-quarters of the responding 
offices are county prosecution offices. 

What prosecution goals are relevant to the MIS project, and what 
applications are planned or in use in prosecutor MIS projects? The ratings 
reported by 100 of the larger prosecution offices responding to the mail sur- 
vey indicated that the highest priority goals relevant to a prosecution MIS 
are control of scheduling and logistics, and allocation of staff based on 
prosecution priorities. Any assessment of PMIS projects should consider the 
capacity and impact of the MIS relative to these goals. Case status reporting 
and workload reports were the two applications most often in use or planned 
by prosecutors, followed in popularity by calendaring and scheduling capa- 
bilities. 

Due to the interest expressed by LEAA, a special telephone survey 
was conducted of PROMIS projects to supplement the data obtained in the mail 
survey. Information was obtained on sites contained on an INSLAW provided 
list as well as sites included in the mail survey. A total of 100 sites, 
thought to be operating, installing or planning PROMIS, were surveyed. Of 
these, 78 sites reported involvement with PROMIS. 

The PROMIS transfer program has been successful in promoting and 
accelerating the adoption of automated MIS capabilities in prosecutors' 
offices. The May 1979 user's conference was attended by over 260 people, 
representing over 75 user jurisdictions in varying stages of PROMIS transfer. 
Since LEAA funding support for prosecution/court information systems now 

strongly favors the use of PROMIS, this program can be expected to grow in the 
future. 

Several versions of PROMIS have evolved, from the original batch 
version to the latest buffered or MAXI version. Substantial modifications 
have been made in certain implementations. It is, therefore, difficult to 
discuss PROMIS as a single system; one must consider the various versions of 
PROMIS in any evaluation. 

More PROMIS systems (taking the various versions of PROMIS as a 
group) are planned or have already been implemented than any other single 
approach. These projects have involved transfer at the software level to a 
variety of hardware and jurisdictional environments. Evidence of the 
increasing trend in the adoption of PROMIS projects was revealed in the mail 
survey. 

Table 4 shows the overall status of the surveyed sites as of October 
1979 broken down by the various PROMIS versions. For the purpose of the survey. 
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PROMIS versions refer to the source of the original software package, and are 
defined as; 

• PROMIS I - the basic batch system provided by INSLAW; 

• PROMIS II - the basic batch system with an on-line inquiry capa- 
bility provided by INSLAW; 

• MINI-PROMIS - unbuffered terminal version, provided by INSLAW; 

• MAXI-PROMIS - buffered terminal version, provided by INSLAW; 

• HYBRID - a combination of PROMIS versions or a PROMIS version 
that has been extensively modified (e.g,, OUSTIS - the large 
scale, on-line system developed by Milwaukee, transferred to 
and modified at other locations). 

Among operational PROMIS projects, the version most often used for 
transfer was PROMIS II, which combines batch updating with on-line inquiries. 
The second most popular operational version is Milwaukee County's OUSTIS, a 
highly modified version of the original batch PROMIS. OUSTIS was modified to 
operate on-line in a shared prosecution/court environment. The version of 
PROMIS most in demand by those prosecutors planning for an MIS is the MAXI- 
PROMIS. 



Table 4. Status of PROMIS Versions 





C Source; Westat Mail/Telephone Survey, 1979 
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f operational system (Sacramento, California) was reoortpri 

for the MINI-PROMIS version (unbuffered terminals) released in December 1978 
Because of the desire for buffered terminals and the tailoriTO Sures 
planners of PROMIS projects have opted for MAXI-PROMIS. ’ 

2. Nature of PMIS Support 

All PMIS components (hardware, software, orocedurpp anH fv/noo 

?h1fsM° system) differ a^ong all JirJsSiclLI’su^ve L"“fo ' 

this study. PMIS support functions also differ, but the primary difference i=! 
funpf"ir®^ functions supported by the PMIS; the purpose of a^PMIS support 
function IS usually the same among the various jurisdictions (e.q. in all 
jurisdictions where the PMIS produces prosecutor's caseload reooris fhneo 
reports ere used by managers as an aid'in assigning S' cSes)" ’ 

fsvnthPai^In^'i°f nature of PMIS support can best be described by combinino 

where MIS functions occur and where performance measures may be made. 

ohaT’oH h jurisdictions visited, 14 of the information systems were 

(or almos^ain^orfh"® . j'^ftice agency, and 11 were shared by all 

Jor almost all) of the criminal justice agencies within the jurisdiction 

f he W^arit ?sfata"s”S%^° Prosecutor’s segment 

cne Mib p It relates to the overall criminal justice system rather than 

confining the analysis to the prosecution functions per se^ 
of prosecution^^*" ^ relationship of MIS capabilities to the stages 

• Events in the process - actions taken and decisions made; 

« MIS activities - functions performed using an automated system; 
o Data recorded - t()e type of information entered into the MIS; 

• Performance measures — measures of effectiveness that can be 

systemrand appropriate data have been entered into the 

• MIS impact — results that can be achieved by effective analysis 
of the performance ipeasures. 

tions are tL'"nn^hpr®^F®r^^f Primary differences in PMIS support func- 
rpn L f functions supported by the PMIS. These differences 

system (Fioure ^ synthesized, integrated criminal justice information 

system (Figure 4) with a prosecutor controlled information system (Figure 5) 
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Exhibit 2. MIS Relationship to Judicial Process 
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Figure 4 represents an integrated system where all criminal justice 
, agencies in the jurisdiction are users of the system; and, the system is 

part of a network that links the local computer with computers at the state 
and national levels. 

Figure 5 represents a prosecutor controlled system where only the 
prosecutor's office has direct access to the PMIS; the prosecutor and other 
f agencies may have access to state and national level systems, but the local 

PMIS is not linked to those systems. 

Some prosecutors believe they are best served by a system under 
their own control. They believe that their data collection procedures would 
be more reliable and they could maintain and improve the programs as needed. 
f The advantages and disadvantages to this form of organization are discussed 

next in Section II E. 

Differences between the integrated and prosecutor controlled systems 
concern, primarily, information exchange: !, data recording , and MIS processing . 

^ Information Exchange . Integrated systems allow the updating of cen- 

tralized records as events occur throughout the judicial system; data are 
immediately available to all system users and could have an impact on decisions 
made at each stage of the process. In the prosecutor controlled system, in- 
formation about the status of individuals, such as outstanding arrest warrants, 
escapees, and persons on probation, is disseminated by the responsible agencies 
^ via manually prepared printed reports. Case tracking, defendant tracking, and 

person tracking is accomplished through the automated system by the prosecutor; 
other agencies requiring updated information must communicate via telephone 
and resort to manual records for data retrieval. 

Data Recording . In integrated systems, user agencies enter their 
^ subject-specific data into the automated system. A prosecutor controlled 

system generally has line prosecutors recording data, or clerks recording data 
from prosecutors' notes. Clerical staff generally enter data on events outside 
their purview (e.g., arrest report data). 

MIS Processing . The integrated system provides both on-line and 
g off-line support to daily operations and management functions of all user 

agencies. The system controlled by the prosecutor provides on-line prdcessing 
support to the prosecutor; of feline processing support is limited to outputs 
that can benefit the operations and management functions of only the prose- 
cutor and the court (schedules and statistics, for example). 

^ The integrated system generates reports to support the operations and 

management of all user agencies. Reports generated by a prosecutor controlled 
system are applicable to the operations and management of the prosecutor's 
office and the courts. The capability exists in both systems, however, to 
generate the same types of reports for the prosecutor and the courts. A list 
of these reports is shown in Exhibit 3. 
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Calendars/Dockets/Schedules 

1. Cases Scheduled for Preliminary Hearing, by; 

• Court/Judge 

• Prosecutor 



2. Cases Scheduled for Grand Jury 

3. Cases Scheduled for Trial, by: 

• Court/ Judge 

• Prosecutor 

• Defense Attorney 

4. Subpoenas/Appearance Notices 

• Witness Supoenas 

• Notices to Arresting Police Officers 

• Notices to Defense Attorneys 

• Notices to Bondsmen 



Case Status 

1. Master Lists of Pending Cases, by; 

• Case Number 

• Defendant Name/Status of Defendant 

• Type of Case (felony or misdemeanor) 

2. Case Aging Lists - showing oldest cases first, by; 

• Court/Judge 

• Prosecutor 

• Pending Grand Jury Action 

3. Disposition Reports, by: 

• Court/Judge 

• Prosecutor 

• Defendant 

• Case Number 

Case Load 

1. Number of Cases Assigned, by:, 

• Court/Judge 

• Prosecutor (individually or grouped by prosecutorial unit) 



Exhibit 3. MIS Generated Reports 
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2. Specially Assigned Cases, by; 

• Court/dudge 

■ Prosecutor 

D. Research and Analysis 

1 . Caseflow analysis 



• Number of Bookings 

• Numbers of Cases Accepted for Prosecution 

• Dismissal Rates/Reasons for Dismissal 

• Conviction Rates by Type of Offense 

• Sentences by Gravity of Crime 

• Probation Rates 

• Incarceration Rates 



2. Work Load analysis 



• Caseload by Court 

• Caseload by Prosecutor 

• Caseload by Type of Offense (felony and misdemeanor) 
3. Defendant analysis, by; 



• Age 

• Sex 

• Race 

f Offense 





Exhibit 3. MIS Generated Reports (Continued) 
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Issues and Assessments 



Research conducted in this study has indicated certain primary areas of 
concern in the development, implementation, and operation of prosecution 
management information systems. These primary areas are; 

• Issues Concerning the Organizational Context of a PHIS Project . 

These issues are concerned with (a) whether the PMIS should be controlled by 
the prosecutor or be part of a state, city or county court or criminal justice 
informati^jn system shared with other criminal justice agencies; (b) resource 
requiremei^^jts for system development and improvement; and (c) the intra- and 
inter-organizational relationships required to effectively manage a PMIS 
project. 

• Issues Concerning the Development and Operation of a PMIS . These 
issues concern (a) the identification of management information requirements; 
(b) the evaluation and selection of alternative hardware and software 
approaches; (c) the staffing required for effective systems development and 
operation; and (d) procedural controls required for effective system opera- 
tion. 

• Issues Concerning the Impact of a PMIS on the Criminal Justice 
System . These issues are concerned with (a) the extent to which PMIS's meet 
their goals and provide their intended capabilities; (b) the impact of the 
PMIS on prosecution/court processes arid personnel; (c) the impact of a system 
on prosecution performance; and (d) the evaluation of such impact. 

« • Issues Concerning the Evaluation of PROMIS Projects . These issues 

are concerned with (a) the comparability and evaluability of PROMIS projects; 
(b) cost effectiveness of PROMIS projects; and (c) expectations of PROMIS 
projects. 

Each issue will be analyzed from three perspectives; first, the issue 
will be defined and discussed; second, where survey data applicable to that 
issue are available, they are summarized; third, the results of all aspects 
of data analysis are interpreted in making a judgmental assessment. 

c 

1. Issues concerning the organizational context of a PMIS Project. 

ISSUE ; Are there significant advantages to the prosecutor control- 
ling his own system, as compared to participating in an interagency shared 
criminal justice information system? 



A crucial issue that has emerged in this study, one that has gener- 
ated wide differences of opinion and wide variations in project organization, 
is to what extent should the prosecutor control the collection, processing 
and reporting of prosecution data? There are at least four types of project 
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organization that have been identified in the literature and observed in the 
field; 

• Computer operations and data base controlled by local 
prosecutor; 

• Computer operations and data base controlled by a statewide 
prosecution/court agency; 

• Computer operations centrally controlled by local government 
data processing agency; data base extensively shared by local 
criminal justice agencies; and 

• Computer operations centrally controlled; data base controlled 
by local prosecutor or court; limited data sharing. 

Among the advantages cited for a self-contained computer facility 
controlled by the prosecutor are assured access and operation, priority on 
programmer expertise, and total security of the case files. Security an 
privacy could be controlled under a system shared by only criminal Justice 
agenciL; however, most prosecutors' offices are headed by elected officials, 
and performance measures can be politically sensitive. 

Polansky points out that county commissioners and state legislators 
are usually not happy about a court (or any other agency in their jurisdic- 
tion) operating an independent facility^ Independent computer operations 
can lead to costly redundancy in hardware and staffing. 

Closely related to the issue of degree of system sharing is the 
issue of resources required for developing and operating the PMIS. Resources 
are limited and one of the major reasons for sharing the system with other 
county or state agencies is to keep the costs down. Most county data pro- 
cessing operations cannot afford a separate criminal justice unit pro- 
gramming or operation. Due to these budgetary constraints the 
have to wait until other possibly higher priority systems (e.g. , tax collec- 
tions) are completed. This can be particularly frustrating when waiting for 

improvements in an existing PMIS. 



Management of the PMIS project is a complex task requiring coordina- 
tion among all units of the prosecutor's office, probably with other criminal 
justice agencies, and possibly with the county or state data processing 

center. > 

A recent Search Group, Inc. research report observed that criminal 
iustice information systems have the potential advantage of avoiding duplica- 
tion of effort by the several agencies involved in the criminal justice 
process. 8 The same report asserts, however, that some questions of propriety 
are raised by the mere fact that courts are using the same data base as police 
and prosecutors, fh these instances, care must be taken to partition data 
sets so that each agency is assured of data pr;i;Vacy and can access only the 






information to which it is entitled. The main point is that avoiding dupli- 
cation of effort, particularly in data recording, avoids errors and therefore 
provides users with better quality data. 



Applicable Survey Data . Prosecutors who reported that they use 
or plan to use PMIS were asked whether access is provided to other criminal 
justice agencies. Responses indicate that 80?d of the systems are shared, 
primarily among prosecutors, courts and police. However, it was evident in 
site visits that the degree of data sharing varies. For example, in one site 
the court receives copies of certain PMIS reports but has its own separate 



court information system. In another site, both the prosecutor and court are 



tied into the same data base and system, each having designated data input and 



access. 



Judgmental Assessment . Formal organization structures may not be 
as important as the informal organization structures and interpersonal rela- 
tionships that evolve. Three PMIS projects were observed where the prose- 
cutor was unhappy with the support from the central data processing facility 
(run by a central county data processing agency). Two of these prosecutors 
were not satisfied with the degree of confidentiality of office records in a 
system not under his control. In these situations, characterized by scarce 
resources and inconsistent priorities between user agencies, prosecutors 
desired their own computer and data processing staff. Such an approach 
usually requires a lower cost computer operation — a minicomputer or purchase 
of outside services. The personalities of the prime movers of the PMIS 
project, be they prosecutors, judges or city/county data processing adminis- 
trators, strongly influence the level of interagency cooperation achieved. 

Extensive data sharing among criminal justice agencies was observed 
in 11 of the 17 sites surveyed. Individual agencies had primary responsi- 
bility for data collection and updating of the PMIS data base at these 11 
sites. Those jurisdictions where extensive data sharing was observed have 
a higher level of interagency cooperation and mutual trust than was observed 
in the limited data sharing jurisdictions. This was true even in non data 
processing activities. Users in extensive data sharing systems have a higher 
level of satisfaction with their systems, particularly when compared to 
jurisdictions having large-scale systems with limited data sharing. Projects 
characterized by extensive data sharing avoid duplication of effort in data 
collection and processing, which results in budgetary savings. In these 11 
jurisdictions where agencies assumed responsibility for updating specific 
items of data consistent with their functional responsibilities, the accuracy 
of the shared data was better than in jurisdictions with duplicate data 
collection procedures. Safeguards on the confidentiality of court/prosecu- 
tion records seemed to be satisfactory. 

If interagency relations permit it, there are significant advantages 
to an integrated criminal justice or prosecutor/court information systems 
approach, These advantages include budgetary savings, data reliability, and 
more effective interagency coordination of criminal justice activities. As 
lower cost and ever more powerful minicomputer based systems become more 
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available, it may become economically feasible for more prosecutors to estab- 
lish computer systems and supporting staffs of their own. However, even with 
minicomputers, staff support costs remain substantial; moreover, the other 
advantages of the integrated system are not realized under this structure. 

2. Issues concerning the development and operation of PMIS. 



3* ISSUE ; To what extent have prosecutors' management information 
requirements and alternative systems approaches been analyzed and defined 
prior to designing the PMIS? 



How does a prosecutor determine which functions should be included 
in a PMIS and what kinds of information are needed? Organizations that have 
been most successful in developing and implementing a useful management 
information system have started with a thorough analysis of their requirements. 
Of the 17 sites surveyed, a thorough analysis and documentation of require- 
ments was noted in only four non-PROMIS and one PROMIS site. Users who make 
the mistake of first deciding to get a computer or deciding to acquire a 
particular software package before they have defined their own requirements 
and resource constraints are likely to become disillusioned with the burdens 
of making the software work and modifying it to meet their own needs. An 
office could survey the applications that are successful in other jurisdic- 
tions. An outside consultant who is familiar with the literature, state-of- 
art, and relevant experience in the field could also be helpful. 



Many publications provide expert opinion on the type of data needed 
by prosecutors. Standards 7/have been developed for case management needs. ^ 
Information needs for caseflow managements docketing and recordkeeping for 
trial court information systems have been documented by other studies.® 



Various models exist which include PMIS applications that have been 
found useful in other jurisdictions. For example, one widely circulated report 
lists various uses of a PMIS by prosecutors.11 



After the requirements analysis and preparation of the system con- 
cepts, the prosecutor's office is ready to consider alternative hardware and 
software systems approaches. The most convenient and economical approach is 
to use existing city/county or state computing facilities and data process- 
ing systems staff, as was the case in 15 of the 17 jurisdictions surveyed. 
There are potential problems in such an arrangement, but many jurisdictions 
(12 or 15 surveyed) have been successful in such arrangements; the success 
of a system does not depend on ownership of the computer. 12 



The use of minicomputers^jl 1 (such as in Boston and Oakland) and 
microcomputersl^ may make prosecutor-owned computer systems more feasible; 
however, the office should still weigh the potential advantages of sharing 
computer facilities and software development and maintenance costs at least 
with _ the court and other criminal justice agencies, as identified under the 
previously discussed issue on project organization. 





Judgmental Assessment . In all but five jurisdictions visited, it 
was difficult to obtain documentation or other evidence that a systematic 
analysis of prosecution information requirements and a feasibility study had 
been prepared prior to proceeding with the development of the PMIS. Few alter- 
natives, if any, to the eventual PMIS approach had been examined. Much of this 
can be attributed to the embryonic state of PMIS applications and the lack of 
data processing systems expertise in most prosecutors' offices. Most projects 
appeared to evolve from a combination of inadequate manpower to cope with large 
caseloads, frustration with court delay and logistical problems, a general idea 
that computers are synonymous with modernization and improved control, sales 
pitches by vendors or consultants, and the availability of Federal funding to 
support the development of PMIS projects. As a result of all of the above, 
unrealistic expectations and unsatisfactory cost-effectiveness characterize many 
projects. 




On the other hand, those PMIS projects that have been based on sys- 
tematic requirements analyses have evolved in phases, adding new applications 
in accordance with a conceptual design and master plan. In these cases, 
expectations have been realistic and have been met. The adaptation or 
tailoring of concepts and/or software transferred from other sites has also 
been accomplished more effectively when the recipient site had a 'well-defined 
statement of its requirements. 



b. ISSUE ; To what extent have prosecutors transferred PMIS's oper- 
ating in other jurisdictions, and what advantages have been observed? 



Various prosecution/court MIS or criminal justice information systems 
exist which have the potential for satisfying a prosecutor's needs. One series 
of software system packages, PROMIS, has been transferred to many types of 
jurisdictions and hardware configurations. 1^ 



There are potential disadvantages to transfer. One is the possibil- 
ity of wasting a great deal of time and money finding out that the system 
being transferred cannot perform the required functions at the new location. 
Most jurisdictions differ markedly in legal procedures, organizations, staff 
capacity, hardware environmment , management style, etc. This means that sqb- 
stantial modifications must be made in transferring a system. One advantag/e 
in developing a system from scratch is that all the capabilities are specifi- 
cally developed for that jurisdiction. Additionally, the prosecutor and his 
staff develop pride of ownership (and resultant enthusiastic use), and they 
are involved in the learning curve from the beginning. Two jurisdictions 
(Chicago's Automated Case Management and Oklahoma's ADRS) report a low cost of 
development from scratch of a prosecution system.6>15 On the other hand, 
such development can be a costly experience if one does not have the proper 
technical staff available. 



Nevertheless, transfer can make good sense not only because of the 
potential savings in time and cost, but because of the uniformity it provides 
for consideration of information requirements. System transfer has been 
encouraged by LEAA funding policies. New LEAA funding policies related to 
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prosecution or court information systems provide grants for statewide PROMIS 
under LEAA's "Incentive Fund Program." This policy encourages multi- jurisdic- 
tional implementation of PROMIS in prosecutors' offices and trial courts.'! 6 

Applicable Survey Data . Fifty (about 57 %) larger prosecution 
offices (over 25 employees) "with operational PMIS projects or planning a PMIS 
indicated PROMIS as their type of system. This suggests a very high interest 
in technology transfer. 

Most prosecutors who undertake PMIS transfer projects do so to save 
time and money. They can also assess their needs better when they can 
observe a well defined model in operation at another jurisdiction. 

Judgmental Assessment . As discussed later in Sections II E.4.b and 
c, little evidence is available from this study to support the notion that 
transfer of PMIS systems has resulted in lower development costs. Unfortu- 
nately, with the current state of the art, many transfer projects have required 
substantial investments in modifying software to fit different hardware/soft- 
ware environments, as well as different organizations, and local rules and 
procedures. Under these circumstances, it can indeed be more costly to invest 
the time and effort to study, test, evaluate, modify and debug an existing 
package. 



The state-of-the-art in software systems packaging, modular design, 
data base management systems, tailoring facilities, pre-compilers, and param- 
eterized report generators has been advancing so rapidly that the picture on 
economics of technology transfer may be undergoing a significant change. Even 
the approach to technology transfer needs a thorough review. It may not make 
sense to think of transferring an entire system, only certain modules, appli- 
cation programs, or software facilities. One should expect to do some special 
programming or modification; the issue is to minimize the extent of it. For 
example, San Bernardino has developed a single interface control program which 
communicates with all application programs. In any transfer to a different 
hardware/software environment, only the interface control program would need 
to be modified. The application programs are all written in a highly transfer- 
able ANSI COBOL. More flexible vendor-supplied software packages, including 
tailoring, report generation, data base management, generalized inquiry and 
statistical packages will also have an impact on technology transfer strategies. 

c. ISSUE ; To what degree has external funding support and technical 
assistance been necessary in PMIS projects? 

Prosecutors generally have very limited financial resources. Is it 
therefore wise to seek external funding, at least for the development of the 
system? The dual problem of obtaining both the funds to develop and install a 
court information system, and to continue operating the system (i.e., institu- 
tionalizing it) was raised in one research report and is a question all 
prosecutors planning as PMIS must confront. 
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Another recent evaluation found that apparently little thought was 
given to local funding of the criminal justice information system after Federal 
funding ran out.'*^ City police agencies were among the users that expended 
more effort as data providers than as data users; hence, they did not feel they 
were benefiting enough to contribute to the cost of operation. This evaluation 
recommended providing management information reports (e.g. , arrest statistics 
by arresting officer's name) in return for data input by the police. Agencies 
which were equally data users and data providers were quite willing to absorb 
a share of operating costs. 

Prosecutors also generally have very limited staff resources. To 
what extent should they attempt to use their own personnel to design and 
develop a PMIS? To what extent should they use outside consultants? A recent 
study reports that both extremes ("Turnkey" or total outside consultant, and 
"Total In-House") possess inherent problems and disadvantages which outweigh 
any possible advantage of either approach. 2 The same study concludes that a 
combination of consultant support with professional in-house staffing is the 
proper solution and recommends outside consultants conduct the initial feasi- 
bility study. 

Applicable Survey Data . Approximately 46 percent of the offices 
using a PMIS reported that a contractor was involved in development of the 
system. Offices that reported using a PMIS were queried on the source of 
funding for their projects. Responses indicate that external funding support 
was necessary in over 80 percent of the 40 operational PMIS projects responding 
The significance of LEAA for development of PMIS's is shown by the data 
indicating that two-thirds of the projects relied on LEAA funding support. 

Judgmental Assessment . None of the sites visited would have under- 
taken their PMIS project without LEAA funding support. Indeed, due to limited 
budgets and staffing, and a natural resistance to automation on the part of 
many attorneys, the rate of adoption of automated PMIS projects would probably 
have lagged considerably had LEAA support not been available. Only the recent 
advances in lower cost computer technology would otherwise have stimulated 
interest. 



The need for LEAA funding is usually associated with technical assist 
ance which the majority of projects utilized. Of the 17 PMIS sites visited, 
ten used an outside contractor, in varying degrees, to advise and/or carry out 
software development. Outside contractors have been used most effectively 
where there is an in-house systems capacity present to properly direct the 
effort and utilize the results. 

d. ISSUE ; Have constraints, such as privacy/security safeguards and 
budgets, inhibited the development and operation of PMIS projects? 

What are the appropriate controls for safeguarding security and 
privacy, and to what extent have they been implemented in the existing systems? 
Federal regulations pertaining to privacy focus on "criminal history record 
information" compiled by criminal justice agencies on criminal offenders. 
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This information is recorded as the result of arrest, detention, or other 
initiation of criminal proceedings. It does not include intelligence, analyt- 
ical and investigative reports, or statistical records and reports in which 
individuals are not identified and from which personal identifiers are not 
ascertainable.^^ The rules and regulations provide specific exceptions for 
certain original records of entry, court dockets, and court records of public 
proceedings. 20 Although these exceptions appear to exempt the courts, court 
managers have been cautioned that the exemptions apparently relate to manual 
records and case files not to automated files created therefrom. This satis- 
fies the definition of a criminal history record. ^ 

Most state and Federal regulations have failed to address the issue 
of when a current court/prosecution record becomes a criminal history record. 
PMIS defendant records with no current cases pending may be purged and placed 
on a criminal history file.'I'l At such time, they may come under the criminal 
history regulations. 

One purpose of the privacy regulations was to eliminate the use of 
FBI rap sheets (criminal histories) which did not record the court disposi- 
tions of arrests. Dispositions must be entered into the files within 90 days 
and most PMIS's provide this capability. Generally, dissemination of automated 
information is restricted to criminal justice agencies. 

There are statutory provisions for sealing or expungement, physical 
security, individual access, and review (including defendants and incarcer- 
ated persons) .^^>21 jhg PMIS should provide logging of all access requests, 
terminal I.D., password controls, physical control, backup, and other tech- 
niques designed to safeguard privacy and confidentiality of records. 1 5,22 

Applicable Survey Data . Reasons for not usdng or planning to use 
a PMIS were solicited and the results reveal that budget and staff are the 
most important constraints for both larger and smaller offices. Next in 
importance was the opinion that there was no need for a PMIS. This opinion 
often appeared to coexist with the budgetary and staffing factors. Privacy 
was considered the least important constraint. 

Judgmental Assessment . Privacy and security constraints have had 
substantially less of an inhibiting effect than budgetary and staffing 
limitations on PMIS development and operation. Nevertheless, in two sites 
visited, there was concern expressed by the prosecutor about safeguarding the 
confidentiality of his records, particularly when the PMIS operation was 
outside of his control. The political nature of the district attorney's 
position makes the distribution of statistical measures of performance highly 
sensitive in many jurisdictions. 

In all sites visited, prosecutors perceive very tight budgetary and 
staffing constraints on their operation. Many of those not perceiving a need 
for a PMIS believe that they have neither the staff nor budgets to even 
consider it. Even those who do plan or use a PMIS would have been unlikely 
to do so had not external funding support and technical assistance been 
available. 
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An important factor in assessing PMIS projects is the extent and 
quality of the available documentation. Sound documentation is crucial to 
proper maintenance of the system, particularly in the event of personnel 
turnover, as well as system transfer. LEAA has observed that two of the 
largest obstacles to be overcome, once the decision has been made to trans- 
fer a system, are inadequate documentation and the fact that certain computer 
languages restrict themselves to one computer manufacturer. 23 Hence, LEAA 
developed a set of special conditions for grants involving automated ’data 
processing. Among other conditions, applicants must agree; 

• To use, to the maximum extent practical, computer soft- 
ware already produced and available without obligation. 

To assist in this area, LEAA publications and regional 
systems specialists should be consulted. 

• That all application programs be written in Federal 
Standard COBOL or ANSI FORTRAN whenever possible. Pro- 
grams may be written in ANSI BASIC for microcomputers 
and minicomputers, subject to certain conditions speci- 
fied by LEAA. 



To provide complete documentation in accordance with 
Federal Information Processing Standards (FIPS Pub. 38). 



n^.pj.j.Lduj,e purvey uaua . njimosc yu percent or offices responding 
to the mail survey indicated they had systems flow, data base and output 
reporting documentation; fewer (74?0 had documented their programs. In 11 of 
the 17 surveyed projects, documentation was available in various forms. How- 
ever, only five non-PROMIS sites had finalized and organized their documenta- 
tion in a fashion suitable for technology transfer. 



Judgmental Assessment . Most PMIS projects have tried to document the 
various components of their system. Many programmers, however, make documenta- 
tion the final task and sometimes never get around to completing it because of 
other crises that develop. Because of the challenge in writing separate tech- 
nical and non-technical manuals for different audiences, a complete set of 
well organized documentation, prepared in several volumes to address management, 
systems/programmer personnel, data entry personnel, operators and user needs, 
is rare. 



PMIS project managers desire to complete their documentation, but 
usually run out of external funding support before documentation is completed. 
As a group, PROMIS projects have above average documentation supplied by 
INSLAW under the LEAA-funded PROMIS transfer program. However, many projects 
have made substantial modifications and these are often not well documented. 
INSLAW has, however, designed some self documenting facilities into its latest 
version, MAXI-PROMIS. 
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3. Issues Concerning the Impact of a PMIS on the Criminal Justice 
System. 

a. ISSUE ; Have PMIS projects specified their goals and intended 
capabilities, and to what extent have these goals been met? 

By the effective use of information, prosecutors should be able to 
meet goals they have set for improved office performance. Before one can 
attempt to assess impact, it is important to assess whether PMIS's are pro- 
viding the capabilities that they set out to provide. A recent evaluation of 
a multi-agency criminal justice information system found that managers were 
expecting much more than they were receiving. Many felt the system only 
supported clerical functions. City officials thought the system primarily 
supported county functions; county officials viewed the criminal justice 
system as primarily a supporter of police functions of cities. The evaluator 
concluded that this was symptomatic of early project over-selling of the 
intended capabilities of the system. 

Applicable Survey Data . Prosecution offices were asked to assess 
the contribution that an ideal PMIS could make to the attainment of certain 
office goals. Those offices that actually had a PMIS were asked to rate the 
expected and actual contributions of their system. 

Table 5 summarizes the comparative contributions of the ideal PMIS 
to the expected and actual contributions of their own PMIS projects. The 
first column shows, on a scale of 0 to 100, an index of the contribution that 
responding prosecutors, on average, believe an ideal PMIS could make to 
attaining office goals. Scheduling and logistical control are perceived as 
the highest area of payoff and an increased conviction rate as the lowest area 
of payoff. Comparison of expected and actual contributions of their own PMIS 
projects reveals a gap, particularly for attaining evenhandedness. On the 
other hand, the gap is smaller for the scheduling goal. 

A large gap between actual and expected performance can lead to a 
sense of frustration on the part of prosecutors. Table 6 summarizes the 
ratio of the perceived actual contribution to the expected contribution as 
well as the ratios between the actual and ideal contributions. The ratio of 
actual to expected contributions indicates that PMIS projects fall short of 
prosecutors' expectations. This is true for both PROMIS and non-PROMIS 
projects. PROMIS projects fare better than non-PROMIS projects in contrib- 
uting to the goal of priority allocations of staff; they are less helpful in 
scheduling and logistical control and improving the conviction rate. On the 
ratio of expected to ideal PMIS contribution, PROMIS projects score consist- 
ently higher than non-PROMIS, which suggests that PROMIS users continue to 
have relatively high expectations for their projects. 
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Table 5. Comparison of Ideal to Expected/Actual Contribution 

of Prosecutor MIS to Management Goals - Jurisdictions 
With 25 or More Employees (median percentage contribution) 



Prosecutor Goals 


Contribution of Ideal 
MIS to Management Goals 
N = 75 


Expected MIS 
Contribution 
N = 73 


Actual MIS 
Contribution 
N = 30 


Allocate staff 
by priorities 


79 


70 


33 


Monitor even- 
handedness 


80 


70 


12 


Control schedules 
and logistics 


89 


80 


73 


Perform research 
and analysis 


81 


74 


33 


Evaluate office/ 
prosecutor 
performance 


75 


60 


24 


Increase con- 
viction rate 


49 


30 


10 



Source; PMIS Assessment Mail Survey, Westat, Inc., July 1979 

MOTE; N's vary slightly for each goal. N's for the "ideal" and "expected" 
columns include jurisdictions operating and planning a PMIS; the "actual" 
column represents only operational PMIS projects. 

Given the ratios shown in Table 6, one would like to combine them 
into a single composite index that could be used in making an overall assess- 
ment of project types. A simple mean of the ratios across all the goals would 
amount to giving each goal equal weight. An alternative, used in this analysis, 
is to weight the ratio of actual to expected for each goal by the relative impor- 
tance weight given that goal, and then computing an average across all goals. 

Two methods of computing the relative importance weights were used and are dis- 
played in Table 6. The first, composite index A, uses each individual respon- 
dent's own goal importance rating as a weight; the second, composite index B, 
uses the mean of the goal importance weights given by all respondents as a weight 
for each goal. The ratio for each goal is then multiplied by the appropriate 
goal importance weight and averaged across all goals to compute each respective 
composite index. On the basis of the individualized goal importance weights 
(Composite Index A), there is almost no difference between PROMIS and non-PROMIS 
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projects in the ratio of actual to expected contributions. Using Composite 
Index B, the non-PROMIS projects have a higher composite index of actual to 
expected contribution. The individualized weights result in a somewhat higher 
composite index for PROMIS than for non-PROMIS with respect to the actual to 
Ideal ratio. Again, weighting each goal the same across the entire population 
® favorable position for PROMIS projects. This suggests that 
ROMIS users have somewhat different goals and expectations than non-PROMIS 
users, and that their actual projects are more in line with their own goals 
and expectations than with the group average. 

One variable that should be controlled in assessing the perceived 
gap between actual, expected and ideal PMIS contributions to office goals is 
the length of time that a system has been operational. Normally, one would 
expect the ratio of actual to ideal to increase over time, as "bugs" are 
eliminated from the system and capabilities are tailored to office needs. 

4 - -.4 1 ^ presents analyses of the ratios of actual to expected, actual 

to Ideal expected to ideal PMIS contributions, controlling for length of 
time the PMIS has been operational. The composite index was computed using 
the individualized goal importance weights defined earlier. 

PROMIS users score higher than non-PROMIS users in the ratio of 
expected to ideal PMIS contribution; i.e., they expect their projects to 
approach the ideal PMIS. This is consistent with the earlier finding for 
Table 6. However, when one compares the ratios of actual to expected or 

the non-PROMIS projects score higher for most goals than the 

PROMIS projects. 

Duplication of effort was observed in eight of the 17 jurisdictions 
surveyed. The degree of duplication, however, varied widely; in two of the 
eight projects (both non-PROMIS sites), card files containing extensive 
duplicative data were being maintained as backup to the PMIS; in two other 
jurisdictions (one PROMIS, one a non-PROMIS site), separate forms (source 
documents) were being prepared for PMIS data input operations, thus data were 
pmJS ^®®°^ded three times (original report, source document, and entry into 
mib;; and, in the other four jurisdictions (one non-PROMIS and three PROMIS 
statistics were being kept manually because users were not satisfied 
with the accuracy of the data. 

judgmental Assessment . A written statement of PMIS goals and 
requirements had been prepared in only five surveyed projects. The prosecu- '' 

tors expectations of PMIS capabilities usually came from contacts with other ■ 

jurisdictions, vendors, outside consultants, magazine articles and profes- 

association meetings. A common method by which prosecutors learn of ; 

PMIS capabilities is through the exchange of information with other prosecu- 
tors. Even if prosecutors do not specify their goals in advance, they ^ 

do have certain expectations of their PMIS projects. f 

The most common applications of PMIS projects are on-line defendant I 

and case tracking or status reporting, workload reports and calendar displays. 
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Table 7. Preceptions of PMIS Contribution 
to Prosecutor Goal Attainment for 
Projects Operational 1-3 years - Juris- 
dictions with 25 or More Employees 




In one jurisdiction, prosecutors have effectively used on-line facilities to 
prepare one case by examining data across several cases for particular 
defendants and witnesses. This information can be used in plea bargaining or 
in discrediting witnesses with prior records. In another jurisdiction, judges 
have used on-line facilities to check de^^ehdant or attorney supplied informa- 
tion in support of continuance motions. 

As prosecutors and their systems staff obtain more experience in 
using their PHIS' s and exchange information, an increasing number of innova- 
tive and more sophisticated applications can be anticipated. For example. 
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Alameda County's DALITE and Los Angeles' PROMIS projects have been attempting 
to use case weighting approaches to assist in determining staff needs. The 
weights reflect the staff requirements of different case types. Several 
informative research designs and findings have been reported by INSLAW using 
the PROMIS data base of the U. S. Attorney's office of Washington, D. C., as 
well as a cross-city analysis of other PROMIS projects. Oklahoma's ADRS 
project has a sophisticated statistical analysis and reporting capability 
which can serve as a model for management analysis applications, \0ther exam- 
ples of innovative PMIS applications can be seen; in Tarrant County, Texas, 
where the prose for various charges are stored on computer files sio that 
charging documents can be rapidly generated whenever charges are changed 
(e.g., by Grand Jury action); and, in Norfolk, Virginia, where arrest reports 
and arrest warrants are automatically generated using on-line booking opera- 
tions. 



Analysis of the mail survey responses and site visits indicate a 
general shortfall in the degree to which PMIS projects have met prosecutor's 
goals. The situation is better with respect to scheduling and logistical 
control applications than it is with research and analysis applications. The 
gap between expected and actual goal attainment will narrow as prosecutors 
gain more experience in using their PMIS. Implementation of new applications 
will help the situation, but it is also anticipated that prosecutor's expecta- 
tions will become more realistic with PMIS experience. 



b . ISSUE ; Have PMIS projects been useful in generati r 
measures, and are they used to improve management and operations?" 



PMIS introduces a capability for performance evaluation that did 
not exist previously. Even though PMIS information is a tool of the prosecutor, 
the existence of information which can be used for evaluation purposes must 
introduce an awareness of performance evaluability which is a new factor in 
prosecution behavior. What is the effect of this new awareness? Is it to make 
the prosecutor more cautious? Does it motivate the prosecutor to adopt partic- 
ular measurable goals and to try to devise strategies and tactics to implement 
these? Does it really motivate and enable more effective prosecution management 
practices? 



More research than ever before is being devoted to improving measures 
of prosecution performance and the quality of justice. A performance measure- 
ment program has been proposed and a preliminary analysis has been made of the 
usefulness of PMIS's in satisfying those requirements. 2^ Recent awards have 
been made by the National Institute for Law Enforcement and Criminal Justice 
to study the area of performance measurement for prosecutors and for courts. 
Various research reports have been published related to the subject of prose- 
cution performance mepsurment, based on data generated by a PMIS. 25 



One problem in developing and interpreting performance measures is 
the wide variation in procedures and terminology among local jurisdictions. 
It could be very illuminating to compare various jurisdictions with respect 
to uniformly-defined performance measures. For example. Church, et al. , 
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recently defined uniform measures of court case processing time in a study of 
court delay .26 gy taking a systematic sample of felony cases terminated in 
21 courts of general jurisdiction, they were able to develop cross-comparable 
measures of case processing time and certain court characteristics. The 
potential advantages of having a nu-her of jurisdictions routinely collect a 
set of uniformly-defined data are very attractive as demonstrated by Brosi in 
a recently released research report .27 

Applicable Survey Data . The mail/telephone survey examined the 
availability of statistical data, for possible evaluation purposes. Table 8 
summarizes those responses. For the types of data specified, PROMTS users 
appear to perceive a greater availability of statistics than non-PRDMIS 
users. Among users, case load statistics are more available than either 
delay or disposition statistics. 



Table 8. Percentage of Sites Reporting Datd 
Available from MIS - Jurisdictions 
With 25 or More Employees 
(N's in parentheses) 



Type of Data 



Case load 



Case disposition rates 



Court delay 



Planning MIS 



Using MIS 



Non-PROMIS 


PROMIS 


Non-PROMIS 


PROMIS 


40.0 


56.3 


72.0 


85.7 


(10) 


(32) 


(25) 


(14) 


45.5 


56.3 


56.5 


78.6 


(11) 


(32) 


(23) 


(14) 


33.3 


48.1 


63.6 


76.9 


(9) 


(27) 


(22) 


(13) 



Source: PMIS Assessment Mail Survey, Westat, Inc., July 1979 



Judgmental Assessment s A majority of the PMIS projects report that 
they generate measures related to workload, court delay, and disposition rates 
of prosecution/court performance." In most of the sites visited where such 
information was available, the prosecution managers were using the statistics 
to manage their workloads, e.g., assigning cases to individual prosecutors, 
prodding judges to move cases faster, ahd developing office plea bargaining 
strategies. PMIS performance measures "have influenced policy development 
and decisionmaking as in the establishment of witness assistance/notification 
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units, career criminal programs, joint prosecution/court projects to reduce 
court delay, special projects aimed at specific types of crime and development 
of uniform charging policies and procedures. 



c. ISSUE : How cost-effective have various types of PMIS proj ects 

been in collecting, pro^ssinq and displaying information? — 



Cost-benefit analyses have proved effective in justifying prosecu- 
tion management information systems. 28 An extensive set of projections can 
be computed on the cost reductions expected through such capabilities as 
automated subpoenas and more efficient use of personnel and/or equipment. 

It has been reported that court personnel often have negative reac- 
tions to the introduction of computer systems. 29 There are, however, certain 
intangible benefits which are difficult to measure, including: 

• Improvement in the adjudication process through more 
effective, efficient and timely operations; 

• Improved public image resulting, hopefully, in improved 
cooperation; and 

• Improved operations and management through research 
results made possible by the availability of data. 7 

Case scheduling and logistical control capabilities are useful for 
both prosecution and court management. Capabilities of the PMIS are also of 
potential use to prosecutors for research, performance evaluation capabili- 
ties and monitoring evenhgndedness in prosecutory and judicial decisions. 

The ability to track cases and defendants from arrest through final court dis- 
position is important to police, prosecution, court and correctional agencies. 
Nevertheless, while PMIS projects prove useful to prosecutors, are they worth 
the costs in staffing and computer time? Are certain types of projects more 
cost-effective than others? 



Applicable Survey Data . Prosecution offices that reported having 
a PMIS were asked to give their development and operation costs. Among 24 
responding jurisdictions with 25 or more employees, the median development cost 
was $150,573. The median monthly operating cost was $3,500. 

The number of offices that have a PMIS and could provide cost data 
in responding to the mail survey was relatively small, making any inferences 
from the sample to the population problematic. Given that caution, Table 9 
reports the average development and monthly operating costs per case screened 
and per case prosecuted in 24 offices that responded to both cost and workload 
questions. The table compares PRQMIS and non-PROMIS projects to the extent 
that deta were available from the mail survey. All else being equal, one 
Would expect projects such as PROMIS, which involve systems transfer, to 
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show lower development costs than other PMIS projects. Howeverj there is no 
support for this theory in the data provided by this limited sample of PMIS 
projects. 



Table 9. Development and Operating Cost 

Jurisdictions With 25 or More Employees 



TYPE OF COST 


NON-PROMIS 


PROMIS 


TOTAL 


DEVELOPMENT 






■' 


- Median 


$150,000 

(15) 


$175,000 

(9) 


$150,573 

(24) 


- Mean cost per 
case screened 


$ 30.32 
(6) 


$ 38.29 
(4) 


$ 33.51 
(10) 


- Mean cost per 
case accepted 


$ 40.79 
(9) 


$ 43.68 
(5) 


$ 41.82 
(14) 


MONTHLY OPERATIONS 


J 




f 


- Median 


$ 3,077 
(15) 


$ 5,000 
(9) 


$ 3,500 1 

(24) 1 


- Mean cost per 
case screened 


$ .39 

(7) 


$ .34 

(4) 


$ .37 

(11) 


- Mean cost per 
case accepted 


$ .50 

(11) 


$ .49 

(5) 


$ .50 

(16) 



Source; PMIS Assessment Mail Survey, Westat, Inc., July 1979 



Judgmental Assessment . The data gathered in Phase I give some indi- 
cation of patterns associated with apparently effective PMIS projects, as dis- 
tinguished from apparently ineffective ones. Some projects were more effective 
in certain aspects of data processing than other aspects; e.g. , some projects 
had apparently effective operational support applications, but inadequate 
management information generation facilities. 



Certain characteristics were observed in PMIS projects with apparently 
cost-effective data processing operations." fine of the most effective project 
situations was one charac^rized by extensive data sharing either in an inte- 
grated prosecution/coutft [^IS or an integrated criminal justice information 
system, supported by ^ffective large-scale central data processing facilities. 
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In most of the effective projects, the prosecutor relied principally on court 
supplied data with the prosecution staff supplying data on attorney assign- 
ments and prosecution witnesses. Assigning the data entry role to the single 
agency responsible for originating the particular data has systematic advan- 
tages in achieving greater reliability and reduced costs (by avoiding dupli- 
cation of data collection effort). 

Another type of project with apparently high cost-effectiveness 
was one based on the use of a minicomputer, under the prosecutor's control. 

To be effective, these projects relied on a few experienced data entry clerks 
who were capable of either directly recording activities in court or coding 
data from the prosecutor's case file. 

A third type of project that was apparently cost-effective was a 
large-scale system that served multiple jurisdictions with remote on-line 
terminals or remote batch data entry facilities for participating agencies. 

The various jurisdictions received apparent economies of scale by sharing a 
competent central systems staff and facilities. 

On the other hand, some types of projects surveyed were not per- 
ceived as cost-effective. One class of such projects is characterized by 
limited data sharing, prosecutor control of data entry, use by a single 
jurisdiction, and use of large-scale computer facilities operated by a 
central city or county data processing unit. In some situations, problems 
were caused by a lack of interagency cooperation and conflicting priorities. 

During^, the site visits, it appeared that sites with limited data 
sharing also had limited interagency cooperation in areas other than data 
processing, while the opposite was observed in sites with extensive data 
sharing. The cost-effectiveness of these projects suffered from unresponsive 
central data processing support for needed system improvements or modifica- 
tions, more burdensome data collection procedures than expected, questionable 
data accuracy, and higher than expected operating costs. 

Another ineffective class of PMIS projects was one characterized by 
a large-scale, ambitious integrated criminal justice information system but 
without either the necessary interagency cooperation and leadership or the 
required central data processing support capacity. 

The 17 PMIS projects surveyed have been effective in supporting 
day-to-day operational data processing needs. Four of these systems, however, 
have not satisfied prosecution desires for information on such subjects as 
crime specific performance measures and disposition rates, but such capabil- 
ities have been planned as a next step in a logical evolution of increasing 
capacity. Thirteen projects have produqed satisfactory management reports. 
However, special problems or crises often arose creating the demand for an ad 
hoc research inquiry. Those projects that had the most effective management 
information reporting capacity usually had generalized and flexible statistical 
and management inquiry software. In two jurisdictions with recent turnover in 
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prosecution management personnel, prosecution managers were not aware of the 
types of PMIS information that wouid be useful to them. In two other juris- 
dictions, the prosecutors were well aware of these needs but were frustrated 
by the delay in getting such information. 

d. ISSUE ; To what extent can the cost, benefits, and impact of 
P MIS projects be objectively evaluated? 

An in-depth evaluation of the impact of a PMIS project must describe 
the project's objectives, activities, intended effects, and the changes that 
occur in the environment. In addition, it should assess whether changes result 
from the PMIS or from other intervening events. 

In their publication, "Intensive Evaluation for Criminal Justice 
Planning Agencies," Weidman, et al. present various designs for conducting an 
in-depth evaluation. They state that many evaluations fail to link program 
activities and results, or definitively describe what actually occurred after 
funds were expended. The collection of information for intensive evaluation of 
prosecutor management information systems may be difficult because much of the 
necessary data may not be available or may be very costly to obtain. 

The availability of baseline data is one consideration in determining 
whether a project's impact can be properly evaluated. Phase I of the NEP is 
designed to document the chain of assumptions linking the expenditure of funds 
to project intervention, the project activities to immediate outcomes, and the 
immediate outcomes to the impact on the problem addressed by the topic area. 

Applicable Survey Data . Descriptive statistics have to be used with 
care. They should be verified and any analysis should be carefully controlled 
to avoid spurious conclusions due to ignoring certain variables. To determine 
the reliability of the responses to the mail survey, a question was asked on 
whether or not the data came from records. The responses ere summarized in 
Table 10 and indicate that caseload data are apparently more available than 
delay data and that some delay statistics are more available than others. 

There is a clear pattern that those jurisdictions using a" MIS were 
able to supply more delay data from records (43.7?o) than those planning a PMIS 
(19.1?o) or those neither planning nor using a PMIS (16.8?o). Among offices 
planning a PMIS, prosecutors planning for PROMIS appeared to acquire their 
delay data from records more often (30-3?a) than those planning non-PROMIS 
projects (7.9?o). 

Judgmental Assessment . The Phase I study of literature and other 
data sources revealed many diverse methods of calculating costs and assessing 
project benefits. These resulted in noncomparable measures among PMIS projects. 
Objective assessments of benefits and impact can be accomplished only if 
uniform definitions of cost elements, performance measures and sound evaluative 
research methodology are applied across projects by a disinterested qualified 
third party. 
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Table 10. Percent Reporting Data Acquired From Records - 
Jurisdiction With 25 or More Employees 
(N's in parentheses) 



No 

MIS 


Planning MIS 


Non- 

PROMIS 


PROMIS 


12.5 


11.1 


28.6 


(16) 


(9) 


(28) 


14.3 


14.3 


33.3 


(7) 


(7) 


(21) 


26.7 


11.1 


34.5 


(15) 


(9) 


(29) 


21.4 


11.1 


30.0 


(14) 


(9) 


(20) 


16.7 


0 


25.0 


(6) 


(8) 


(20) 


9.1 


0 


30.4 


( 11 ) ^ 


(6) 


(23) 


81.8 


66.7 


65.2 


I ( 11 ) 


(3) 


(23) 



Delay Data 

Arrest To; 

Filing 

Indictment 

Trial verdict 

Misdemeanor guilty plea 



Felony guilty plea 
related to indictment 

Nolle/dismissal 



Case Load Data 

Intake and dispositions 81.8 66.7 65 2 

('ll) (3) (23) 

Source; PMIS Assessment Mail Survey, Westat, Inc, July 1979" 



Using MIS 

Non- 

PROMIS PROMIS 



extracting data from records and in conductino key informant inter 
views. Reasonable estimates can be made to fill in gaps. 

vxds sumcient observations, and statlstical1„rt^%1r?ir‘va\SZ 
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characteristics, court characteristics, policy and other variables that might 
affect performance simultaneously with the PMIS. This requires an in-depth 
analysis beyond the scope of the Phase I site visits. 

Measurement of PMIS effectiveness and impact is quite different from 
the usual program evaluation situation. PMIS px'ojects have an advantage in 
that they can be self-monitoring; i.e., they generate the prosecution/court 
performance measures that, in turn, can be used to measure PMIS impact. A data 
base can be developed from PMIS case/defendant history files to show changes 
over time in a time series format. 

One problem that occurs in many program evaluations is the unavail- 
ability of reliable base line data (conditions existing before the program 
intervention). However, PMIS project evaluations have an advantage over other 
types of project evaluations in the collection of baseline data in that the 
nature of the PMIS project is a chain of events: 

(1) Install PMIS; 

(2) PMIS measures prosecution/court activity; 

(3) PMIS users review statistical reports; 

(4) PMIS users make policy and operational 
decisions; and 

(5) Repeat cycle, starting from (2) above. 

The PMIS-generated measurements of prosecution/court activity (Step 2) would 
need to precede the use of that information and any impact on prosecution/court 
performance. Thus, the data collected in Step 2 constitute a set of baseline 
data. However, since prosecution/court users must assimilate the technology 
of the PMIS and learn how to use it effectively, the impact of the PMIS is not 
expected to be immediate. 

Some effects of the PMIS project may result from the development 
effort itself and the associated staffing and organization activities. The 
baseline data for analyzing these effects can usually be gathered from key 
informant interviews, and reviews of office records and procedures. Manual 
samplings of court case records also can be conducted to fill in data gaps in 
the PMIS data base and develop baseline data for the period preceding the 
PMIS operation. 




A PMIS that provides its intended capabilities may still fall short 
of having any impact on the prosecution/court process. For example, the PMIS 
can enable prosecution management to monitor evenhandedness. Beyond monitoring, 
however, is the attainment of evenhandedness itself. It is possible that 
PMIS, especially in the presence of particular prosecutorial policies, creates 
greater evenhandedness in charging, plea bargaining, trial outcomes, and in 
sentencing. This potential PMIS impact should be systematically investigated 
in any in-depth evaluation. 
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Some published research reports demonstrate that the PMIS can be 
employed to evaluate aspects of police and judicial performance. 23 Recid- 
ivism studies have obvious implications for the corrections process as well 
as for prosecution and sentencing policies. 

Judgmental Assessment . The Phase I effort revealed no instances 
where the impact of an MIS on the prosecution/court process had been ade- 
quately measured. Any impact assessments in Phase I are necessarily judgments. 



About one third of the 17 project sites visited appeared to have 
utilized their PMIS's to a sufficient extent that the project might have a 
measurable impact on prosecution or court performance. Fifteen projects 
reported improvements in case scheduling and logistical control; nine projects 
reported reduced backlogs and improved rates of disposition. 

Prosecutors participating in the PROMIS cross-city analyses have 
focused on reducing attrition (losses of cases through declinations, nolles 
and dismissals), or on dropping cases earlier to conserve resources. Some 
prosecutors may be able to achieve increased effectiveness in dealing with 
certain crimes and certain types of offenders, e.g. , career criminals, with 
assistance from their PMIS projects. Through more systematic use of their 
PMIS capabilities, some jurisdictions may be able to show an increase in the 
proportion of arrests that end in convictions. 




Only two sites were observed which systematically reviewed measures 
of evenhandedness. One site did compare disposition rates across prosecutors 
to attempt to evaluate individual performance and flag an imbalance in the 
proportions of pleas, trials, nolles and dismissals. 

In two situations, the PMIS was reported to have a negative impact 
on prosecution/court performance by siphoning off scarce manpower to prepare 
computer inputs rather than process cases. These PMIS projects received only 
limited use and could be expected to have no discernible impact. 

4. Issues Concerning the Evaluation of PROMIS 



a. ISSUE ; 
non-PROMIS projects? 



Are PROMIS projects more comparable and evaluable than 



One significant advantage expected from PROMIS projects has been that 
the uniform concepts, common terminology, data definitions and other common 
features would make these projects comparable and thus permit cross-city 
comparisons. To the extent that common terminology and procedures were used, 
there might be economies of scale in the availability of technical assistance, 
research and development and education/training programs. 



Applicable Survey Data . During the site visits, the typical site 
generated adequate operational and management statistics on a periodic basis. 
Three non-PROMIS sites had available a general purpose statistical software 
package which could be used in special evaluative analyses. The typical 
PROMIS project has such a statistical software package. 
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Another important dimension in examining the evaluability of PMIS 1 

projects is the range of applications in use. These have been recorded for I 

all sites visited. I 

In reviewing applications designed to support operational control, | 

it was observed that case, charge and defendant tracking applications are : 

common for lOOSo of the PMIS projects surveyed. There is a drop-off to 80 ?d | 

in calendar displays and persons tracking capacity (witnesses, victims, I 

attorneys, etc.); and witnesses notices are produced by only of the j 

surveyed sites. On-line booking is non-existent, of course, among non-inte- } 

grated projects, but used in seven out of 10 of the integrated systems. * 

\ 

I 

A review of applications designed to support management uses of j 

the PMIS reveals that case inventories are produced at all 17 survey sites. | 

Workload reports are produced at 83% of the sites, while case aging (at 30%) | 

and case weighting (at 12%) are much more rare. The latter features consti- I 

tute a method of estimating manpower requirements based on case complexity; l 

i.e., the case is given a different weight based on the degree of complexity j 

or other attributes which affect staffing requirements. PROMTS projects | 

score substantially higher (83%) than non-PROMIS projects (30%) in crime- 1 

specific and attrition analysis capacity. In general, a higher proportion of I 

PROMTS projects in the sites visited supported management information needs I 

than did the non-PROMIS projects (e.g. , across the six PROMTS sites, 70% of | 

the management type applications were produced; across the 11 non-PROMIS ; 

sites, 38% were produced). ! 

Judgmental Assessment . Experience among PROMTS user jurisdictions 
has been that wide diversity still exists in terminology, data elements and 
procedures. Many of these differences are built into the legal and political 
structure of each jurisdiction and will not be changed by any information sys- ; 

tern. Substantial software changes have been made in modifying PROMTS in most 
jurisdictions in which it has been implemented. This requirement has been j 

recognized by INSLAW in its latest MINI and MAXI versions of PROMTS which high- 
light a tailoring facility to simplify modifications of data definitions, file 
structures, etc. 

INSLAW produced a cross-city analysis of performance measures in 13 
jurisdictions using PROMIS. This effort took over two years to accomplish, 
although not on a full-time basis. On the scale carried out, it is a commend- 
able achievement, especially important for its focus on prosecutorial discre- 
tion and crime specific analysis. Even though each jurisdiction used PROMIS, 
the technical and dcta manipulation tasks were very difficult due to all the 
variations. There have been other cross-city analyses not relying on PROMIS; 
for example, the National Center for State Court's study of delay in 21 
general jurisdiction courts26 and a revealing study of the judicial process 
in three cities by Jacob and Eisenstein.^2 jp view of the potential for 
cross-city analysis, the surface has only been scratched by these publications. 

Questions remain as to the special need for PROMIS projects in such studies, 

as opposed to inclusion of other PMIS's, or the use of survey techniques by I 

independent statistical agencies. I 



59 




What is special about the INSLAW study is its focus on the exercise 
of prosecutorial discretion, the potential for crime specific analyses, and 
the working relationship built up between INSLAW and the district attorneys of 
the participating jurisdictions. This relationship probably could not have 
been developed without the mutual experience of the LEAA-funded PROMIS transfer 
program. Over the years, INSLAW personnel have demonstrated sensitivity to 
prosecutors' concerns, competence in their technical fields, and understanding 
of the prosecution court processes and the pitfalls of inter-jurisdictional 
comparisons. If funding is continued, the cross-city program may result in 
the creation of a unique multi-jurisdictional time series data base for 
analysis of prosecutorial discretion, court performance, policies, and impacts 
of various types of interventions. 

As a group, PROMIS projects make more use of management and statis- 
tical information than comparable non-PROMIS projects. Many of them have had 
cost and benefit studies performed by INSLAW in assessing the feasibility of 
their adoption of PROMIS projects. As a result of the above considerations, 
and common conceptual models, the PROMIS projects as a group should be easier 
to evaluate than non-PROMIS projects. 



effective than non-PROMIS projects? 



One of the assumptions LEAA has made in supporting technology trans- 
fer is the potential for saving development costs and improving systems quality 
and effectiveness. After more than six years of funding PROMIS transfer proj- 
ects, it seems appropriate to explore their cost-effectiveness relative to 
non-PROMIS projects. 



First, it is fair to recognize the extreme challenge inherent in 
attempting to develop a portable PMIS. There is great diversity among juris- 
dictions in their prosecution and court systems. Legal procedures, court 
organization, statutory constraints, management philosophy, resources, and 
staff capacities vary to such an extent that many question the basic concept 
of transfer at the software level. The ability to economically and conve- 
niently adapt or revise data definitions, data entry formats, data processing 
procedures and output displays is essential for software portability. 

Applicable Survey Data . Of 13 operational PROMIS projects, the 
median development costs are in the range of $100,000 to $199,000. As shown 
in Table 9 above, this is comparable to the costs for non-PROMIS projects as 
compiled from the mail survey. 



One possible explanation for the lack of substantial development 
cost savings in transferring PROMIS is the extent of software modifications 
needed. Table 11 presents a tabulation of the opinions of PROMIS site person- 
nel on the degree of difficulty experienced in transferring PROMIS. More than 
half (13 of 24 agencies) considered transfer to be more difficult than expected, 
while none of the agencies considered it to be easier than expected. It is 
noteworthy that when one compares th§ different versions, the MINI- and MAXI- 
PROMIS versions have a much higher ratio of "more difficult than expected" 




O 



60 




implementation experience. Of the 13 projects reporting that the transfers 
were more difficult than expected, seven attributed the difficulties to basic 
software problems, such as program bugs, as shown in Table 12. 



Table 11. Difficulty Experienced in Transferring PROMIS 
Operational and In-Transfer PROMIS Projects 



( 



( 



C 



c 



c. 




Source; Westat Mail/Telephone Survey, 1979 
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Table 12. Cause of Difficulty in Implementing 
Operational and In-Transfer PROMIS 
Projects for Which Data Are Available (N=13) 




The survey data indicate that those who have operational PROMIS proj- 
ects are generally satisfied, although this situation is less so for PROMIS II 
users. 

Judgmental Assessment . From the limited data collected on operational 
projects in Phase I, there is little support for the theory that PROMIS projects 
are more cost-effective than non-PROMIS projects. The development costs do not 
appear to be less, possibly because of the necessity for substantial software 
modifications to accommodate inter jurisdictional differences, including among 
other factors, software/hardware incompatibilities. It may be true that future 
PROMIS transfers will be less costly than those to date because there are more 
hardware and software configuration versions from which to choose. As shown in 
Tables 6 and 7 above, there is also no evidence that PROMIS projects have been 
more effective than non-PROMIS projects, except possibly in providing manage- 
ment and statistical information. 



ISSUE; What are the expectations for PROMIS projects? 



With the extensive promotional materials and support available, it 
is interesting to ask three questions about the expectations of PROMIS users. 
First, are their expectations unrealistic, leading to projects that are too 
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ambitious with resultant disenchantment and low cost effectiveness? Second, 
and related to the first question, how do the expectations of PROMIS users 
compare with non-PRCMIS users? Third, what changes are occurring in the de- 
velopment and use of PROMIS projects, and what expectations are realistic 
for the future? 

Applicable Survey Data . Tables 6 and 7 compared expectations of 
PROMIS with non-PROMTS projects. Table 13 provides information on PROMIS proj- 
ects either desiring or planning changes. Of 23 operational or in-transfer 
PROMIS projects responding, 14 or 61 percent either desired or planned changes. 
The table shows that only 3 (27%) of 11 PROMIS I and II projects plan to stand 
pat. The largest swing is to MAXI-PROMIS; 7 (30%) of the 23 projects are 
switching to that system. 

Judgmental Assessment . With no clear evidence of cost and effective- 
ness advantages, why have so many prosecutors opted for PROMIS projects? It 
is apparent, from the Phase I site visits and telephone survey, that the type 
of user who is most enthusiastic about PROMIS projects is the chief prosecutor, 
that is, each jurisdiction's district attorney. INSLAW has enjoyed success in 
reaching top management in prosecutors' offices; this is by no means a trivial 
accomplishment. The PROMIS User's Group has been successful in providing for 
cross-city communication and exchange of insights on prosecution management 
policies, use of PROMIS management reports, and funding opportunities. The 
Phase I survey found that a relatively high proportion of PROMIS users experi- 
ence more difficulty than expected in transferring and implementing their 
systems, especially with software problems and data collection burdens. At 
the same time, from responses to the mail survey, it was apparent that prose- 
cution managers held high expectations for their PROMIS projects. 

One of the contributing factors to high expectations for PROMIS 
appears to be LEAA's continuing funding for PhCMIS development. Indeed, since 
implementation of PROMIS, like implementation of many new software packages, 
often takes longer than planned, PROMIS transfer completions have occasionally 
been outpaced by the release of enhanced versions of the system. For example, 
the highly publicized MINI-PRCMIS package was released by INSLAW for field 
installation in December, 1978. In a telephone survey taken during September 
and October of 1979, which included sites listed by INSLAW as planning or having 
PROMIS projects, only one operational MINI-PROMIS as found. While this 9-month 
lag from release to operation is not in itself surprising, the survey revealed 
that very few of the sites were still planning to implement MINI-PROMIS. Many 
were switching to a still newer version, MAXI-PROMIS. 

MAXI-PROMIS, the latest version of PROMIS, was released by INSLAW 
in a preliminary stage in August 1979 for field installation. Of seven sites 
contacted who were in process of transferring MAXI-PROMIS, all reported major 
software modifications were needed. Systems personnel at these sites reported 
that they were doing extensive software development and modification to make 
MAXI-PROMIS work. As of October, 1979 no sites were found where MAXI-PROMIS 
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was operational. The system is still new and undergoing change. Howbver, 
expectations remain high for the many attractive features of this new version, 
e.g., the tailoring and self-documentation features. 




Table 13. Changes Planned or Desired 
Operational and In-transfer 
PROMIS projects (N=23) 




Source: Westat Mail/Telephone Survey, 1979 
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III. FEASIBILITY TESTS 



A. Introduction 

1 . Evaluability of PMIS Projects 

An LEAA report suggests three conditions which must be met for 
projects to be evaluable. The conditions are; 

• Users of evaluation results must agree on definitions 
of activities, the conditions to be changed, and the 
kinds of expected outcomes. 

• The key project assumptions must be stated in terms 
which can be tested objectively. 

■ Program or project managers must clearly define at 
least one use for evaluation information in making 
a decision or in initiating administrative action. 

Few projects surveyed in the first segment of Phase I had explicit 
statements of their goals and expected outcomes. However, there were enough 
models available through transfer projects and other well conceived PMIS 
projects to infer a set of implicit prosecution goals from interviews and the 
literature search. The evaluation framework, developed from observations in 
the first segment of Phase I and discussed in Section II C and in Volume I, 
presents what our generalizations of definitions of activities, conditions to 
be changed, and kinds of expected outcomes. 

Although the key project assumptions linking PMIS implementation to 
expected impacts were not explicitly stated in any project, the study enabled 
the research team to infer an implicit chain of assumptions. The evaluation 
framework also specifies both quantitative measurements and judgmental assess- 
ments by independent observers as a means of objective evaluation. 

The final evaluability condition, identification of a clear use for 
the evaluation results, is the most significant factor. As stated earlier, 
large sums have been invested at the Federal, state and local levels in devel- 
oping and operating PMIS projects. Interest in PMIS development is expected 
to continue. However, particularly in light of recent Federal budget uncer- 
tainty, Federal, state and local government decisionmakers must allocate 
increasingly scarce funds to competing program areas. In the past, many PMIS 
decisions have been mdde based on intuitive, emotional, or political motiva- 
tions, rsther than objective information. In the future, funding decision- 
makers and prospective PMIS project managers and users will almost certainly 
require a more complete understanding of the characteristics of successful 
PMIS development and operation, the situational variables that encourage or 
hinder successful PMIS imjalementation, and the expected results, costs and 
impacts. 
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Analysis of the data collected during the first segment of Phase I 
indicated that all three conditions for evaluability appeared to be present 
and data required to measure performance/impact of PMIS projects appeared to 
be available. Thus, testing the feasibility of a Phase II evaluation design 
for PMIS projects was initiated. 



Purpose of Feasibility Tests 



Due to the large number of projects surveyed in the first segment 
of Phase I, only a limited amount of time could be spent on each. Even where 
site visits were made, the project team had to rely on available reports and 
documentation or information gathered in interviews with a few key personnel 
on site. Thus, it was not possible to elicit detailed information considered 
crucial for a definitive evaluation of PMIS projects. One purpose of the 
feasibility tests, then, was to determine the actual availability of data 
needed for in-depth evaluations, and, where study team and/or PMIS project 
resources permitted, to collect that data for the purpose of testing evalu- 
ative methods. 



Once data were collected in sufficient detail, tests were conducted 
to determine feasible methods of measuring and assessing PMIS impact on the 
prosecution and judicial processes, PMIS cost-effectiveness, and PMIS transfer 
potential. 




Within the context of the general evaluation framework, the evalu- 
ator's specific approach was conditioned by two factors beyond his control; 
the age of the PMIS being evaluated, and the resources available to conduct 
the evaluation. Therefore, the project team tested the feasibility of both 
a priori and a posteriori evaluation designs (to be applied, respectively, to 
relatively new and relatively old PMIS installations), and the feasibility of 
evaluation approaches involving both intensive and non-intensive data collec- 
tion. 

For relatively new projects, the ^ priori design provides for 
thorough measurement of baseline conditions, and monitoring of changes in 
performance during the evaluation data collection period as measured by pre- 
defined indicators of success. For projects that have been operational two 
or more years, the a posteriori design focuses on the construction of a time- 
series from historical data files in a retrospective analysis of changes over 
time. At sites where system enhancement is occurring, these two approaches 
can be combined. For example, impact of operational support applications may 
be evaluated on an a posteriori basis while a newly introduced management/ 
statistics capability may be evaluated on an a priori basis. 

Each design perspective has advantages and disadvantages. On 
recently implemented projects, it is easier to reconstruct an accurate set 
of measurements of baseline conditions. Memories are fresher for inter- 
views on situational variables; judgmental observations can be made, and 





66 



manual records on court performance are more readily available. The disad- 
vantage in a prior i design is the limited time frame available for observing 
changes. For <^>cample, on newly implemented projects, only about six months 
of operation ciin be observed during the course of the feasibility study. Of 
course, this time frame will vary somewhat depending on the date the particular 
project became operational. 

The a posteriori design has the advantage of providing about up to 
24 months of data in a time series format. Such data are amenable to con- 
structing more valid evaluative research designs to measure project impact. 

To the extent that gaps exist in the definition of baseline conditions, it is 
more difficult to reconstruct those from sources external to the PMIS itself 
because of failing memory, warehoused court records, etc. However, a properly 
constructed ^ posteriori design can yield the data necessary to measure 
project impacts. 

The ^ priori and posteriori approaches involve the use of both 
PMIS data and samples of manual records. Resource constraints on the feasi- 
bility study precluded intensive data collection and analysis at all evalua- 
tion sites. Therefore, both intensive and non-intensive data collection 
approaches were used. 

The intensive approach, attempted in three sites, incorporated 
collection and analysis of PMIS-generated time-series data on indicators of 
success, as well as sampling and analysis of manual records to attempt to 
compute performance measures and to assess the quality of PMIS data. The 
non-intensive approach involved, instead, an assessment of the availability of 
PMIS time-series data to support impact evaluations and of the availability 
and comparability of manual records to support analysis of PMIS data quality. 
While use of the intensive approach at all sites would have been desirable, 
the non-intensive approach nonetheless generated valuable insights into the 
feasibility of PMIS evaluation. 

4. Test Sites 

Six PMIS projects were used as test sites for determining the 
feasibility of PMIS evaluations. Sites were chosen because of their charac- 
teristics and a preliminary indication of being an evaluable PMIS project.* 
Figure 6 depicts the division of PMIS projects by classes. This set of 
classes constitutes an initial division of PMIS projects for assessment of 
general characteristics of each group. The feasibility study was not intended 
to provide sufficient information for generalizing class characteristicsj 
rather it was designed to test methods for developing such information. The 



*Selection criteria for feasibi^litj'r; test sites and comments relevant to 
their selection are contained in Volume II, Final Report. 
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class numbers shown below are indicated by the numbers in the appropriate box 
in Figure 6. The sites selected within each class were; 

• Class 1 - Boston (Suffolk County), MA - Case Management 

System (CMS); 

• Class 2 - Oakland (Alameda County) , CA - District Attorney 

^ Automated Legal Information System (DALITE); 

• Class 3 - Portland (Multonomah County), OR - MAXI-PR0MI5 

• Class 4 - Golden, CO - Colorado Regional PROMIS (MAXI-PROMIS) 

( m Class 5 - Norfolk, VA - Total Recall Adult Criminal Element 

Record (TRACER), a regional system; and 

■ Class 6 - Oklahoma City, OK - Arrest Disposition Reporting 
System (ADRS) , a statewide system. 

C The PMIS classes, the sites selected to represent each class, and 

the data collection/evaluation approaches are summarized in Exhibit 4. 



B. Availability and Quality of Data 
C 1 . Data Availability 

Survey teams spent three days collecting data at non-intensive sites 
and five days at the intensive sites. The primary survey teams consisted of 
two persons for each site visit. The survey teams visiting Norfolk and 
Oakland were each augmented by a Westat field representative. These field 
( representatives were trained interviewers/ data abstractors employed to test 

the feasibility of abstracting case/defendant information from manual records. 

A data collection instrument was prepared for gathering information 
at the test sites. The collection instrument consisted of a separate section 
for each agency or office (prosecutor, courts, police and data processing 
C facility) from which information was sought. Face-to-face interviews were 

conducted in each office or agency. The collection instrument was used as an 
interview guide and used to record data that were extracted from records or 
opinions expressed by the respondent. Exhibit 5 lists the general categories 
of information covered by the collection instrument and shows the availability 
of data within each category from the agencies (sources) at the six test 
C sites. As indicated in Exhibit 5, most data sought in general information 

categories were either obtained during the surveys or it was determined that 
collection of such data is feasible. 
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DATA COLLECTION APPROACH 


EVALUATION APPROACH 


PMIS CLASS 


SELECTED SITES 


Intensivel/ 


Non- 

Intensive^/ 


A Priori!/ 


A Posteriori^/ 














1 . Small minicomputer 


Suffolk County, MA 
(CMS) 




X 


X 


X 


2. Large minicomputer 


Alameda County, CA 
(DALITE) 


X 






X 


3. Large scale computer 
limited data sharing 


Portland, OR 
(MAXI-PROMIS) 




X 


X 




4. Large scale computer, 
extensive data sharing, 
prosecutor/court 


Golden, CO 
(PROMIS/MAXI-PROMIS) 


X 






X 


5. Large scale computer, 
extensive data sharing, 
integrated CJIS 


Norfolk, VA 
(TRACER) 


X 






X 


6. Large scale computer, 
extensive data sharing, 
multi-jurisdiction 


State of Oklahoma 
(ADRS) 




X 




X 



1 / Collect data for time series analysis. 

IJ Determine availability of data for time series analysis. 
Z/ ^ priori - before PMIS implementation. 
y fli posteriori - after PMIS implementation. 



Exhibit 4. Data Collection/Evaluation Approach 



o 

i 

Exhibit 5. Availability of Data by General Category 


1 

i 

|. 

j 

1 

! 

5 


1 

i 

1 . data can su^:.“ed 


e 

t Category of Data Expected From 

Test Sites (Outline of Collection 
Instrument) * 


Actual Data Availability Status 


i 


Time Series Data 


Norfolk 


Oakland 


Oklahoma 

City 


Boston 


Portland 


i 

Golden 


; availability of time series data is shown in Eyhihif < 


TRACER 


DALITE 


ADRS 


CHS 


PROMIS 


PROHIS 


i f-" three sources, (l -pMS h to 


PR 


CT 


PO 


DP 


Prosecutor 


PR 


SAC 


DP 


Prosecutor 


PR 


DP 


PR 


DP j 


1 




A. Personnel 

Number assigned 


Y 


Y 


Y 


Y 




FS 


Y 


Y 


Y 


Y 


Y 


Y 


1 

Y 1 


2< Salaries 


Y 


V 


Y 


Y 


Y 


FS 


Y 


Y 


Y 


Y 


Y 


Y 


Y ! 


1 ■■ r ■« “ ”K 


/ B. Equipment 

1. Hakes, models, quantities 


Y 


Y 


Y 


Y 


■Hi 


NA 


Y 


Y 


Y 


Y 


Y 


Y 


Y 


2. Costs 


Y 


Y 


Y 


Y 


Y 


NA 


Y 




Y 


Y 


Y 


Y 


Y 


■ 


outputtin7ai; hi=“ o7Ica7 f L”?he - 

1 t- "-“S -J 

because of time and money^constraints”^^!^ evaluation methods, 

! appropriate data could be obtained from PMirfilL^arall’tS^sitL^^^^ 

1 '' 

1 

i 

1 

i 

i . 

: ’ •) 

' i 
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C. Input to PHIS 
1. Methods 


Y 


Y 


Y 


Y 


HI 


NA 


Y 


NA 


Y 


Y 


Y 


Y 


i 

Y ! 


2. Volume j 


Y 


Y 


i 


Y ! 




NA 


F 


F 


Y 


FO 


FO 


FO 


FD ‘ 


0. Output use and utility 


Y 


V 


Y 


NA 




FS 


Y 


NA 


Y 


FO 


NA 


FO 


NA ’ 


E. Data Quality 

1. User judgments 


Y 


Y 


Y 


NA 


^H 


FS 


Y 


NA 


Y 


FO 


NA 


FO 


NA 1 


2. Comparison of PHIS data 
with manual records 


Y 


Y 


Y 


Y 


Y 


NA 


F 


F 


F 


FD 


FO 


FO 


FO i 


^ F- Prosecutor goals for PHIS 

1. Identification of Goals 


Y 


NA 


NA 


NA 


IHI 


FS 


NA 


NA 


Y 


Y 


NA 


Y 


NA 


2. Prosecutor ratings of PHIS 
contributions to goal 
attainment 


Y 


NA 


NA 


NA 




FS 


NA 


NA 


Y 


FO 


NA 


FO 


NA 


G. Interventions 


Y 


Y 


Y 


Y 


Y 


FS 


F 


F 


Y 


F 


N 


Y 


! 

^ ! 


H. System operations 

1.. Methods and procedures 


Y 


Y 


Y 


Y 


HI 


NA 


Y 


Y 


Y 


Y 


Y 


Y 


i 

Y i 


2. Costs 


Y 


Y 


Y 


Y 


Y 


NA 


Y 


Y 


Y 


FO 


FO 


FO 


FO 1 

4 


I. Monthly caseload/caseflcw 
statistics 


Y 


Y 


NA 


Y 


N 


1 NA 


F 


F 


F 


FO 


FD 


FO 


j 

FO 1 


3. PHIS development 
1. Methods 


NA 


NA 


NA 


Y 


IH 


NA 


Y 


Y 


Y 


Y 


Y 


Y 


.1 

Y 


2. Costs 


NA 


NA 


NA 


Y 


Y 


NA 


Y 


Y 


Y 


Y 


Y 


Y 


1 1 

1 Y ] 


K. Transferability 

1« Hardware/software 
characteristics 


NA 


NA 


NA 


Y 




1 

NA 


Y 


Y 


Y 


NA 


Y 


NA 


Y 1 


. 2. Documentation status 


NA 


NA 


NA 


Y 


Y 


NA 

1 


Y 


Y 


Y 


Y 


Y 


Y 


1 


LEGEND: Y : Yes, data obtained 

N = No, date not available 

F = Feasible to collect data, not obtained at non-intensive aites 
FO = Feasible to collect data when system is fully operational 
FS e feasible to collect data, needed on a statewide baais 
NA z Not applicable to agency, no attempt made to collect data 

'Agencies: PR z Prosecutor; CT z Court; PO z Police; DP z Data processing facility; SAC z Statistical Analysis Center 
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Exhibit 6. Availability of Time Series Data 





Site Location/PMIS Acronym 




Norfolk 


Oakland 


"T 

Oklahoma 

City 


Boston 


Portland 


Golden 


Data expected from test sites 


TRACER 


DALITE 


ADRS 


CMS 


PROMIS 

1 


PROMIS 


A. Computer history tapes 


! 

Yes 


Yes^^^ 


M 2/ 

No- 


No 


No 




B. PMIS generated statistical reports 
for the past 24-36 months 


Yes 




Partial 


No 


M 5/ 
Nor- 


No 


C. Manually prepared statistical 

reports for the past 24-36 months 


Yes 


No 


Partial 


No 


No 


No 


D. Manually maintained case files 


Yes 


1 

Yes 


Yes 


Yes 


Yea 


Yes 


E. Prospective PMIS generated reports 
probably obtainable 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 



— History tapes from HP system could not be processed on IBM systems. 

2 / 

~ History tapes exist and could probably be obtained, but would require authorization from prosecutors statewide. 

3/ ' 

— History tapes were made available from original PROMIS system; new PROMIS not yet operational. 

A/ 

— No statistical reports have been generated from DALITE; statistics generated by CORPUS are considered unreliable. 

PROMIS not yet operational; statistical reports are provided to the court in computer listings generated by the State 
Judicial Information System. 
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The feasibility of abstracting information from manually maintained 
case files, for the purpose of collecting time series data for periods prior 
to PMIS implementation, was also considered. Testing of abstraction proce- 
dures was conducted at Norfolk and Oakland. The time and cost involved in 
the data abstraction procedures are shown in Exhibit 7. The cost of extrac- 
ting data using these procedures averaged $1.57 per case. 

Once needed data are abstracted, processing of that data is required 
to produce statistics for specific time periods (monthly, for example) to 
facilitate time series analysis. To accomplish this processing by computer, 
it would be necessary to enter the data into a computer record and then 
process that data to generate desired statistics. The cost estimated to 
accomplish this processing (see Exhibit 8) is estimated at $.87 per case. 
Computer programming to accomplish this processing would be a one-time esti- 
mated cost of about $350 if "canned" statistical packages (such as SAS or 
SPSS) are to be used.* 

The cost to develop time series data for a jurisdiction such as 
Boston (where statistics are not available for the preceding 24-36 months), is 
estimated to be from $14,640 (24 months - 6,000 cases) to $21,960 (36 months - 
9,000 cases) based on approximately 3,000 cases per year at a cost of $2.44 
per case (for abstracting, entering, and processing the data). Considering 
the labor force for such an operation, the data abstraction on 6,000 cases 
would require approximately 215 person days; for 9,000 cases, 322.5 person 
days. 

Cost-Benefit Data 



The purpose of collecting cost-benefit data was to determine the 
feasibility of performing cost-benefit analysis of systems representing the 
six PMIS classes. There was no intent to develop a model for cost-benefit 
analysis for this study as several models were available**, and others probably 
have been used for predicting cost or benefits in jurisdictions not included 
in this study. It seems appropriate that if an analysis is to be performed 
for a PMIS for which cost or benefits have been predicted, the model used for 
those predictions should again be used for that jurisdiction to permit compar- 
isons of common data (for before and after PMIS implementation), and for 
validating the model. 

Exhibit 9 shows the availability of cost-benefit data at the test 

sites. 



* Programming costs will be substantially higher unless the analyst has 
structured the case abstract as a fixed-length record. 

**INSLAW cost-benefit model; MULTNOMAH COUNTY OREGON cost-benefit package; 
National Center for State Courts cost-benefit methodology for Evaluation 
of State Judicial Information Systems. 



Exhibit 7. Abstracting Data from Manually Maintained Case Records 



Items regarding test sites 


Norfolk 


Oakland 


Totals 


1. 


Number of records from which data were 
abstracted 


67 


73 


140 


2. 


Total time (minutes) needed to abstract 
data (time needed to retrieve case file 
not included) 


1,124 


i 

1,278 


2,402 


3. 


Average time (minutes) per record (case) 
needed to abstract data 


16.8 


17.5 


17.2 


4. 


Average wage rate ($ per minute) of 
person abstracting data 


$ .075 


$ .108 


$ .091 


5. 


Average cost per record (case) to 
abstract data 


$1.26 


$1.89 


$1.57 



Exhibit 8. Estimates Regarding the Processing of Abstracted Data 



Items involved in processing 


Estimates 


t 

Average time (minutes) needed to enter on record (case) 




I into PMIS 

E 


5.0 


2. Average wage rate ($ per minute) for data entry clerk 


$ .060 


3. Average cost per record (case) for data entry 


$ .30 


4. Average monthly computer processing cost to produce all 




monthly statistics needed for time series analysis 


$97.00 


5. Average number of cases processed monthly as input to 




statistical reports 

f.'* 


5,491 


6. Average cost per record (case) for computer processing 




of monthly statistical data 


$ .57 


7. Total estimated cost per record (case) to process 




^ , abstracted data (data entry + computer processing) 


• 

CD 
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Exhibit 9. Availability of Cost-Benefit Data 

Page 1 of 3 

^ Site Location/PMIS Acronym 



Cost-Benefit Items 


Norfolk 


Oakland 


0klah>ims 

City 


Boston 


Portland 


Golden 


TRACER 


DALITE 


ADRS 


CHS 


PROMIS 


PROMIS 


A. Cost Items 

1. Total development 
costs for PHIS 


$222,240i/ 


$265,000 


$248 ,00olZ 


$137,785 


1 $140,000 

! 


$406,4161/ 


a. analysis, design, 
programming 


$ 54,10o4/ 


$ 42,185 


$217,51oZ'' 


$ 58,985 


$136,900Z/ 


- 


b. documentation 


$ 18,532 


- 


- 


$ 15,000 


1 


- 


c. equipment purchase 


$ 63,360 


$222,815 


$ 30,490 


$ 63,800 


$ 3,100 


- 


d. other (testing, 
training, etc.) 


$ 86,2484^ 


1 ** 


- 


- 


- 


- 


2. Local prosecutor's share 
of operating costs 


$ 30,419^ 


! 

1 $155,821 


n/aZ/ 


$ 33,600 


$ 65,67b5/ 


- 


a. personnel costs 
(salaries and 
fringes) for local 
prosecutor's office 


Yesi/ 


$ 44,878 


$ 84,986^/ 


$ 27,300 


I 

$ 21,740^/ 




b. equipment leasing 


Yes 


$ 7,080 


$ 31,46ll'' 


N/A 


$ io,i5o5/ 




c. equipment 
maintenance 


Yeoi./ 


$ 23,642 


$ 2,7755./ 


N/A 


$ 8,ooo5/ 


- 


d. communications j 


Yesi' 


N/A 


$ 19,,2455./ 


N/A 


$ 12,28b5/ 


- 


e. other operations 

costs (depreciation ' 
of equipment, 
supplies, technical 
assistance) 


1 

1 

2/ 

$ 4,584^' 


1 

1 

1 

1 

$ 80,221^'' 


$ 24,26i5./ 


$ 1,200 


i 

1 

$ 3,OOo5/ 




f. computer processing ‘ 
(charge for CPU time| 


See below 


n/aI/ 


! 

$ 87,2725./ 


n/aZ/ 


$ 10,50o5/ 


- 


(1) on-line 
processing 
(inquiries/ 
entry) 


$ 10,61o5/ 


NoZ/ 


Fi/ 


NoZ/ 


Foi/ 


fo5/ 


(2) batch 

processing 

(output 

reports) 


$ 12,345§.'' 


NoZ^ 


Fi/ 


NoZ/ 


Foi/ 


FO§./ 



Legend: NA = Not appiicable 

C No = Data not available 

Ves = Data available 

F 3 Feasible to obtain data, not collected at non-intensive sites 
FQ = Feasible to obtain data after system is Fully operational 

1 / 

— Regional systems: TRACER (Norfolkl serves multinie criminal Justice agencies in multiple jurisdictions. 

-yPROMlS (Golden) serves prosecutors in multiple jurisdictions. ADRS (OKLAHOMA) is a statewide system. 

•jyFurther breakdown of costs for individual items is available. 

XyOperating costs were taken from current budget documents ($27,539) plus the estimated annual cost for data entry ($2,080) 

( TyEsclmated for first year of operations. 

— PHIS serves prosecutors sLatewide; figures refer to cost of PHIS operations as they apply to the Statistical Analysis 
Center (SAC) of the Oklahoma Crime Commission as estimated in latest budget request. 

— Norfolk ADP facility charges users according to CPU time. In determining chargeout rates, Norfolk indicates users share 
- of personnel and equipment costs which equate to the costs for batch and on-line processing as shown on lines 2f (I) and (2) 
■gyHinlcomputers surveyed do not maintain a record of CPU time used for various transactions. 

— Central ADP facilities using large scale computers maintain records of CPU time usage by transaction; it is therefore 
^.feasible to obtain t lese data. 

-gyEstimates not appLicable to prosecutor, but can be made by other criminal justice agencies, 
j — ^'Figures represent three year average. 
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Exhibit 9. Availability of Cost-Benefit Data (Continued) 



Page 2 of 3 



Site I ocat. lon/PMI S Acronym 





Norfolk 


Oakland 


OkJ nhoma 
City 


Boston 


Portland 


Golden 


Cosl-Benofit Items 


TRAftR 


DALITF 


ADRS 


CHS 


PROMIS 


PROMIS 


B. Benefit items - annual 
sav i nqs 

1. cost displacement 
(personnel, equip- 
ment or supplies no 
longer a recurring 
expense) 


See 

examples 


None 

indicated 


None 

indicated 


None 

indicated 


See 

examples 


None 

indicated 


Cx amples: 

• eliminate second 
(night) shift of 
clerks 


None 

indicated 








$ 49,000 
estimated 


! 


• eliminate filing 
cabinets, card- 
vendor, VJStU- 
triever, etc. 


Estimates- . 
possible^' 






1 


$ 480 

estimated 




® reduce office 
supplies needed 
for manual records 
that are replaced 
by PMIS operations 


Estimates , 
possible^' 






1 

1 

1 


Estimates 
possible 1 




2. Cost avoidance 

(personnel, equipment 
costs not expended due 
to increased produc- 
tivity or efficiency) 


See 

examples 


1 

See 

examples 


None 

indicated 


1 

i 

1 

See ' 

examples ' 


1 

See 

examples 


See 

examples 


Examples: 

• PHIS generation of 
reports, warrants, 
subpoenas, etc. , 
reduces clerical 4 
typing time 


Estimates 

possible 


None 

indicated 




None 

indicated 


$ 12,000 
estimated 


Estimates 

possible 


■ reduce or eliminate 
need to period- 
ically increase 
staff to cope with 
,, increased caseload 


»» 

Estimates-. 

possible^.' 


1 

j 

1 

None ' 

indicated 


i 

1 


None 

indicated 


$ 25,36oZ5/ 

estima*' ed 


None 

indicated 


• reduce the need to 
obtain answers to 
case or defendant 
Inquiries 


Estimates 

possible 


Estimates 

possible 




Estimates 

possible 


' 

$ 8,000 
estimated 


Estimates 

possible 


• reduce or eliminate 
need for periodic 
procurement of 
additional equip- 
ment to cope with 
increased caseload 


Estimates 

possible9/ 






None 

indicated 


$ i,7oo 15/ 
estimated 


None 

indicated 



Legend: NA = Not applicable 

No = Data not available 
Yes = Data available 

F = Feasible to obtain data, not collected at non-intensive sites 
FO = Feasible to obtain data after system is fully operational 



if^Regional systems: TRACER (Norfolk) serves multiple criminal justice agencies in multiple jurisdictions; and 

„ .PROHIS (Golden) serves prosecutors in multiple jurisdictions. ADRS (Oaklahoma) is a statewide system. 

4yurther breakdown Of costs for individual items is available. 

^yOperating costw were taken from current budget documents ($27,539) plus the estimated annual cost for data entry ($2,880). 
.^^Estimated for first year of operations. 

— PHIS serves prosecutors statewide; figures refer to cost of PHIS operations as they apply to the Statistical Analysis 
, .Center (SAC) of the Oklahoma Crime Commission as estimated in latest budget request. 

— Norfolk ADP facility charges users according to CPU time. In determining chargeout rates, Norfolk indicates users share 
...of personnel and equipment costs which equate to the costs for batch and on-line processing as shown on lines 2f (1) and (2) 
•B-^inicomputers surveyed do not maintain a record of CPU time used For various transactions. 

— Central ADP facilities using large scale computers maintain records of CPU time usage by transaction; it is therefore 
-.feasible to obtain these data. 

^^Estimates not applicable to prosecutor, but can be made by other criminal justice agencies. 

— Figures represent three year average. 
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Development costs were taken from grant documents and records that 
account for expenditures for various development components such as analysis, 
design, and programming. 

Operating costs were taken from current budgets for those systems 
that were fully operational at the time of the survey (Norfolk, Oakland, 
Oklahoma City, and Boston). Estimates of operating costs were taken from 
a cost-benefit package prepared by Portland. Operating costs for the old 
PROMIS system in Golden were obtained, but have not been included in Exhibit 
9 which refers to new PROMIS. 

Benefits could be identified for the fully operational PMIS proj- 
ects and predictions of the value of benefits have been made for the two 
PROMIS projects (Portland and Golden). Portland used the Multnomah County 
cost-benefit model to predict benefits while the INSLAW developed cost- 
benefit model was used for Golden. The model used by Portland defines 
benefits in general terms allowing for the recording of specific benefits in 
an open-ended manner. This format is conducive to illustrating the avail- 
ability of cost-benefit data at the test sites and has been used as the 
outline of benefits listed in Exhibit 9. 

Exhibit 9 shows the cost savings predicted by Portland. The 
comment "estimates possible" indicates that a fairly accurate cost savings 
can be attached to the defined benefit; "judgment possible" indicates that 
cost savings attached to the defined benefit would represent a "best guess" 
by experienced personnel; and "none indicated" means that there was no 
indication that the defined benefit is applicable to the PMIS surveyed. In 
general, value estimates are more generally available for benefits in the 
"cost displacement" and "cost avoidance" categories than in the "value added" 
category; therefore, cost/benefit evaluation becomes more judgmental as the 
mix of benefits includes more of the latter group. At one end of the spectrum, 
Portland, anticipating several types of cost displacement and cost avoidance 
benefits, has completed a cost/benefit analysis largely in financial terms. 

For Oakland and Boston, and particularly for the Oklahoma statistical system, 
the benefits are almost completely of the value-added type. The fact that 
this makes cost/ benefit analysis of these systems more subject to judgment 
does not imply that the systems are less desirable than the others. 



Data Concerning Transfer Potential 



A number of environmental and system factors may influence the 
transfer potential of a PMIS. Exhibit 10 shows the data collected at the 
test sites regarding PMIS environment and characteristics. 



- Detailed cost data on the analysis, design, and programming compo- 
nents of PMIS development were not available at all sites: Oakland and Boston 

did not identify these costs; and Oklahoma City, Portland, and Golden could 
calculate these costs fairly accurately. Norfolk, on the other hand, had 
complete cost details on all developmental components. Since any jurisdiction 
considering acquisition of a PMIS by transfer will incur costs relating to all 




V' 




79 



Exhibit 10 . PMIS Environment and Characteristics 

Page 1 of 2 



Site Location/PMIS Acronym 



Environment and Characteristics 
Descriptors 


Norfolk 


Oakland 


Oklahoma 

City 


Boston 


1 

Portland 


Golden 


TRACER 


D.ALITE 


ADRS 


CHS 


PROMIS 


PROMIS 


A. Environment 

1. Area served by PMIS 


Region 


County 


State 


County 


County 


Region 


2. Population served 


1,250,000 


1,100,000 


2,766,000 


723,080 


1,000,000 


1,500,000 
















3. Number of local 
prosecutors 


16 


120 


316 


106 


60 


25 


4. Local prosecutor's 
annual caseload 


2,900l^ 


3,aool^ 


50,000 


3,000l'^ 


12,200 


2,000l'^ 


5. Agency controlling 
computer operations 


City AOP 


Prosecutor 


County ADP 


Prosecutor 


County ADP 


County ADP 


6. Extent of PMIS usage 


muitiple 
agencies and 
jurisdictions 


prosecutor 

only 


multiple 
agencies and 
jurisdictions 


prosecutor 

only 


prosecutor 

only 


prosecutors , 
of multiple 
jurisdictions 


B. Characteristics 
1. Date MIS became 
operational 


7/76 


10/74 


1/77 


11/77 


not . 

operational— 


1/80 


2. Method of PMIS 
development 


in-house/ 

contractor 


In-huuse/ 

contractor 


in-house 


in-house/ 

contractor 


in-house/ 

transfer 


in-house/ 

transfer 


3. Cost of development 


$158,880 


$ 42,165 


- 


$ 73,985 


$136,900 


$172,722 


4. Cost of equipment 


$ 63,360 


$222,815 


- ’ 


$ 63,800 


$ 3,100 


$233,694 


5. Total developmental cost 


$222,260 


$265,000 


$248,000 


$137,785 


$140,000 


$406,416 


6. Annual operating cost 
for entire PMIS 


$270,768 


$155,821 


$250,000 


$ 33,600 


■i 

$ 65,678 


- 


7. j/Lpcal prosecutor's 
annual operating cost 


$ 30,619i'' 


$155,821 


i 

NA ' 


$ 33,600 


$ 65,67si(' 


- 


8* Hardware 

a. size of computer 


Large 


Mini 1 


Large 


Mini 


Large 


Large 


b. central processing 
unit 


IBM 370/145 


HP3000/III 


IBM 370/158 


Nova 2/10 


Amdahl 270 


Honeywell 662C 


c. terminals used by 
prosecutor 
(make-type) 


IMS - CRT's 


HP - CRT'S 


RACAL - HILGO 
CRT's 


Data j 

Terminal, 

Inc. - CRT's 


IBM - CRT's 


Haieit me 
CRT's 



i^elony ^ases . 



2 / 

— For the purpose of this study, a PMIS was considered operational if data were being entered and if outputs (visual displays 
in response to inquiries and/or printed reports) were being produced at the time of the survey. 

—'^Operating costs were taken from current budget documents ($27,539) plus che estimated annual cost for data entry ($2,08d). 

4/ 

— Operating costs are estimates for the first year of operation, taken from the cost-benefit package. 
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Exhibit 10 . PMIS Environment and Characteristics CContinued) 



Pag.o . 2 ... o f_.2. 

Site Location/PMXS Acronym 



Environment and Characteristics 
Descriptors 


Norfolk 


Oakland 


Oklahoma 

City 


Boston 


Portland 


Golden 


TRACER 


DALITE 


ADRS 


CMS 


PROMIS 


PROMTS 


9. Software 

a. operating system 


DOS/VS 


MPE - III 


DOS 


HICOS - II 


0S/VS1 


GEOS 


b. data base 

management system 


CICS - VS 


IMAGE 3000 


IMS 


None 


AOABAS 


I-D-S 


c. teleprocessing 
monitor 


CICS 


MTS 3000 


CICS 


None 


COMPLETE 


TDS 


d. programming 
language 


ANSI COBOL 


ANSI 

COBOL 74 


ANSI COBOL 


BASIC 


ANSI 

COBOL 68 


ANSI 

COBOL 74 


10. Input method 


on-line 


on-line 


on-line by 5 
counties, 
forms mailed 
by others, 
then on-line 
by SAC 


on-line 


on-line 


on-line 


11. Processing capabilities 
a. on-line inquiry/ 
response 


Yes 


Yes 


Yes 


Yes 


Yes 


Planned 


b. 6( 'line printing 
(e.g., forms, 
notices) 


Yes I 


Yes 


I 

Yes 


Yes 


Yes 


Planned 


c, on-line report 
generation 


Yes 


Yes 


No 


Yea 


Yes 


Planned 


d. off-line report 
generation 


Yes 


Yes 


Yes 


No 


Yes 


Yes 


12. Types of support and 
reports 

a. on-line booking 


Yes 


No 


No 


No 


No ! 


No 


b. on-line arrest 
reports 


Yes 


No 


No 


No 


No 


No 


c. on-line warrants 


Yes 


No 


No 


No 


Yes 


Yes 


d. schedules 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


e. workloads 


Yes 


No 


Yes 


Yes 


Yes 


Yes 


f. statistics 


Yes 


No 


Yes 


Planned 


Yes 


Planned 


13. Documentation status 
a. general system 
description 


Complete 


Complete 


Complete 


Complete 


Complete 


Complete 


b. system design 


Complete 


None 


Complete 


None 


Complete 


Complete 


c. program source 
listings 


Complete 


Conplete 


Complete 


Complete 


Complete 


Complete 


d. logic flow charts 


Complete 


None 


Complete 


None 


Complete 


Complete 


e. operational manual 


Complete 


None 


Complete 


Complete 


Complete 


Complete 


f. users manual 


Complete 


None 


Complete 


Complete 


Complete 


Complete 



V/ ~ 

— Felony cases. 



2 / 

— For the purpose of this study, a PMIS was considered operational outputs y^ere being produced at the time of the survey. 
—^Operating costs were taken from current budget documents ($27,539) plus the estimated annual cost for data entry ($2,880), 

4/ 

— Operating costs are estimates for the first year of operation, taken from the cost-benefit dockage. 



aspects of system development, it would be helpful to be able to compare costs 
of each development component between transfer systems and those fully devel- 
oped at a jurisdiction. 

2. Judgmental Assessment of Data Quality 

Crucial Feasibility Issues 

The data collection effort in the field investigations included 
three types of tasks. First, the project itself was described, e.g. , data 
on staffing, costs and computer facilities. Second, the MIS was described, 
including its operational capacity and interventions over time, its inputs, 
and its outputs. Third, it was desirable to collect sufficient data on the 
operation of the criminal justice system, e.g. , monthly caseload and dispo- 
sitions, to permit the construction of a time series data base. 

As expected, much of the data collection was straight forward. The 
three types of data that were most difficult to obtain were comparable project 
costs, external checks on the accuracy of computer data, and time series data 
on criminal justice system performance. Thus, the assessment of feasibility 
was most concerned with these three data collection issues. 

Given the collection of adequate time series data, another major 
issue was the availability of appropriate techniques for analyzing the impact 
of the prosecution MIS. These techniques had to be able to measure changes in 
criminal justice system performance in the time frame of the PMIS interven- 
tions. In addition to detecting changes in performance patterns, techniques 
were needed to assist in attributing any changes to the PMIS interventions. 
Thus, data were required to describe significant variations in caseload mix, 
personnel, policies, organization, facilities, and other factors that could 
affect performance in addition to the PMIS itself. 

Commonality of Data Across Test Sites 

Common data were available from all test sites regarding; 

• Personnel - their salaries, fringe benefits and their work- 

load relevant to PMIS operations 

• Equipment - makes, models, quantities, cost of purchase or 

lease qnd maintenance costs 

• Operation of PMIS - methods of input, processing, and output; 

usey utility, and quality of outputs 
(from fully operational sites); hardware 
and software characteristics; and status 
of PMIS documentation 

• Development of PMIS - methods of development. 
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Data lacking commonality among sites includes: 

• Caseload - caseload data differs within jurisdictions as well 

as among jurisdictions. In Norfolk, for example, 
caseload statistics maintained by the prosecutor 
represent "adult felony indictments"; in the 
Circuit Court, the caseload figures represent 
counts (there may be several counts in an indict- 
ment) of felonies and misdemeanors; and the District 
Court statistics represent the number of hearings 
held. Caseloads do not necessarily repx-esent work- 
load, as far as PMIS operations are concerned. In 
all test sites except Norfolk, the prosecutor (or 
SAC in Oklahoma City) is responsible for entering 
all data pertaining to a case or defendant. In 
Norfolk, the police enter arrest data and the courts 
enter disposition data. It is important, then, to 
define the manner in which caseload counts are made; 
if cost per case is a factor to be considered, a 
common denominator for counting "cases" should be 
used. 

■ Cost of PMIS Development - developmental cost data vary not 
only in degrees of detail, but also in components 
included (e.g., contractor cost only or in-house 
staff as well) and treatment of "soft" costs (e.g., 
management efforts by chief prosecutor). Moreover, 
when comparing development costs per case, report, 
inquiry, population, and prosecutors served across 
jurisdictions, it is important to' cake note of the 
area served by the overall PMIS. In systems serving 
regional (Norfolk) and statewide (Oklahoma City) 
areas, it is difficult to allocate the cost of 
development for one specific prosecutor's office. 
Therefore, in some sites, developmental costs were 
stated in gross terms and not equated to the support 
provided one prosecutor's office. In Golden, on the 
other hand, the grant application shows expected 
development costs for each district. 

• Operational costs - total costs of operation can be determined 

for each PMIS. Cost backup data leading to those 
totals vary among PMIS projects. Centralized ADP 
facilities, such as Norfolk, Portland and Golden, 
charge prosecutors for ADP services. In Norfolk, 
the prosecutor is charged for his share of PMIS 
operations based on his usage of CPU time. The rate 
per CPU minute is set to cover the costs of ADP 
personnel, ADP equipment, and ADP equipment depre- 
ciation. In Portland and Golden, charges are made 
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according to CPU time used plus the cost of personnel 
time. In Oklahoma City, SAC budgets for PMIS opera- 
tions and individual prosecutors are not charged for 
the service. Cost of operations for prosecutor owned 
systems (Oakland and Boston) involve primarily direct 
costs for personnel, equipment rental, and supplies 
devoted to the PMIS. In the latter cases, realistic 
costs per transaction (input, inquiry, and report) 
could not be determined within the available time 
and resource constraints, thus precluding the com- 
parison of such costs across all test sites. 

3. Comparison of Expected Versus Actual Availability of Data 

It was expected that caseload and caseflow statistics would be more 
readily available; if not being produced by the PMIS, it was expected that 
manually kept records would be on hand. Monthly statistical reports are 
currently PMIS generated only by Norfolk and Oklahoma City; such reports are 
planned for Boston, Portland, and Golden. Only one manually prepared statis- 
tical report (from the Norfolk Circuit Court) was available. 

More cost-benefit data were expected to be available. Cost savings 
have been predicted for only two sites using cost-benefit models; none of the 
other four sites have documented either predicted oi\. actual savings. 

\\ 

It was expected that more data would be available regarding new- 
PROMIS operations in Golden. INSLAW newsletters and contacts with site 
personnel indicated that the system was "operational." \As used by INSLAW, 
"operational" means that the software has been installed' on the hardware 
system, and data are being entered into the systems. At ih® time of the site 
visit, between 500-700 cases had been entered into the Golden new PROMIS 
system and inquiries could be made on those cases; however^, no scheduled out- 
put reports were being produced. Also, the proportion of cases entered to 
date was so small that the system was not being used in dayito-day operation. 
Except as indicated above, the availability of other data wap as expected. 

C. Analysis of Test Data 

1. Cost -Benefit Analysis 



Quantifying PMIS costs is a relatively easy task compared to quan- 
tifying PMIS benefits. Only Portland identified a reduction in prosecutor's 
staffing and equipment that is expected because of PMIS implementation; 
these benefits, if reductions actually take place, can be easily quantified. 
None of the other prosecutor's offices could identify such clearly defined 
benefits. In Norfolk, however, the police and the courts are active, on-line 
users of the system and those agencies can identify and quantify extensive 
benefits directly related to the PMIS. 
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One important PMIS benefit identified at all sites is the improved 
capability to track cases. The fact that cases will no longer "fall through 
the cracks" (an expression used repeatedly), or "fewer bases are dismissed 
because of age," are improvements indicated by the prosecutor. The PMIS query 
and response capabilities reduce the burden of manually retrieving case and 
defendant information, a PMIS benefit of concern to line prosecutors and the 
support staff. More efficient use of personnel time, a result of improved 
scheduling using PMIS capabilities, is another benefit often cited by prosecu- 
tion personnel. 

Benefits derived from new reports, which would not be economically 
feasible without the PMIS, are difficult to define and may be impossible to 
quantify in many jurisdictions. Portland has used the method of estimating 
the cost of manually producing new reports and subtracting from that the 
estimated cost of PMIS report generation to determine a cost savings. This 
method may be valid for certain types of reports (for example, statistics to 
satisfy state reporting requirements) , but other factors should be considered 
in determining the value of new reports. A report, although similar or iden- 
tical in content, may be used differently among jurisdictions or even within 
a jurisdiction. In Norfolk, for example, the Circuit Court used TRACER case 
aging and caseload reports to aid in identifying problems regarding the 
time involved in case processing; the result was that less court time is now 
being spent on felony 1 and 2 cases. Similar benefits were not identified 
in the prosecutor's office, although his office receives similar outputs. 
Utilization of the TRACER outputs differ between the prosecutor and the 
court; and the value placed on those outputs would therefore differ between 
the two agencies. In the prospective IN5LAW cost/benefit analysis prepared 
for Golden, this difficulty is addressed by use of "willingness to pay" as 
the evaluation measure; however, even that measure is difficult to assess in 
advance of PMIS implementation. 

Benefits can be identified and quantified where PMIS outputs replace 
manually generated products such as warrants, subpoenas, witness notices, and 
scheduled reports. Clerical/secretarial personnel can give fairly accurate 
estimates of the time required to manually produce such items, so costs of 
manual production can be determined by knowing the wage rate of the producers. 
The CPU time required to by the PMIS to produce these items can be determined 
and costs derived from the rate charged for CPU usage. A valid estimate of 
cost savings can be made for those functions actually replaced by PMIS 
operations. 

Benefits change over time. In the Norfolk situation, discussed 
above, the benefit of decreasing case processing time occurred approximately 
20 months after PMIS implementation; the benefit resulted from the initiation 
of new reports as part of a phased plan for implementing PMIS support. 

Other factors will effect the time when PMIS benefits occur: 

% In Portland, for example^ dual operations (PMIS and manual) 
are planned for a 6-9 month period, and dual operations 
took place in Boston for an extended time. 




• A "learning curve" will be involved for prosecutors who 
' receive management-type reports never before provided to 

them, and benefits will accrue only after experience has 
been gained in the utilization of these reports. 

■ Benefits expected or benefits actually accrued may be 

reduced or negated by management decisions made after the 
! PMIS has been operational for some time. In Norfolk, for 

example, two police booking stations were consolidated 
into one because of the on-line booking capability of 
TRACER, resulting in a substantial savings in manpower 
and equipment. Now, because of transportation problems 
and fuel consumption, the police are re-opening the second 
C. booking station, thus reducing, to a large degree, a PMIS 

benefit. Other examples include jurisdictions that continue, 
or revert to, manual recordkeeping because of inaccuracies 
in PMIS outputs, although the PMIS continues to generate the 
outputs; the PMIS generated report may have originally been 
considered a benefit, but has emerged as an added cost in 
(' such situations. 

In those jurisdictions where technical resources are limited (Oak- 
land and Boston, for example), benefits are slow in developing because PMIS 
personnel must devote full time to daily operations. In both jurisdictions, 
the PMIS is primarily a case/defendant tracking system with printed outputs to 
( support scheduling, case assignments, and retrieving of information. Both 

jurisdictions plan to have statistical reports but neither jurisdiction will 
reap benefits from such outputs for some time. The quantification of benefits 
at these sites would depend, primarily, on the value judgments of the prosecu- 
tors because little in way of personnel and equipment cost savings can be 
identified. It was interesting to note, however, that the Oakland PMIS had 
C an impact on legislation* and that the Boston PMIS has proven helpful to the 

police**, which are examples of other benefits that would require judgments 
to quantify — or possibly cannot be quantified at all. Even though very few 



*DALITE was used by the Alameda County prosecutor's office to aid in the 
research of cases involving a "Ballard Motion" which requires rape victims 
to submit to psychiatric treatment prior to trial. Cases that involved a 
motion (the Ballard Motion, per se, could not be identified by the system) 
were selected and listed by the PMIS. The prosecutors performing the 
research then retrieved file jackets for only those cases on the list, pre- 
cluding the need to look at all file jackets for motions. The research 
revealed extensive delays in trial proceedings because of the Ballard Motion. 
The resulting court decision on the matter led to repeal of the Ballard 
Motion legislation. 

**0ne Boston police investigator useS^ the PMIS generated schedule of cases to 
identify persons scheduled to appear in court who are "friends of fugitives." 
On the day of trial, the investigators visit the court to see if the fugi- 
tive shows up as a spectator at his friend's trial. A number of fugitive 
arrests have been made using this procedure. 
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quantifible benefits can be identified for these systems, the prosecutors are 
enthusiastic about the improved capabilities provided by the PMIS and, at 
this point, certainly consider the system to be worth the cost. 

The main points emerging from this analysis of cost-benefit data 

• A PMIS may provide benefits to more than just the prosecutor, 
even in those jurisdictions where the prosecutor has sole 
access to the system, and particularly where the system is 
shared by other criminal justice agencies. An analysis of PMIS 
costs and benefits should, therefore, include the examination of 
capabilities and outputs provided to all agencies concerned. 

• A number of benefits can be identified that may be difficult 
or impossible to quantify. Some benefits can be quantified 
merely by comparing the cost of manual versus PMIS functions; 
while other benefits require a judgment of their value by the 
users, for which there is no standard measurement. 

• The results of a cost-benefit analysis may be invalidated by 
subsequent events that violate the assumptions of the analysis 
and affect the magnitude of a given cost or benefit element. 

2 . Performance Measur es 

Of the six sites selected for field investigations, two were con- 
sidered as ^ pro^ri evaluation sites because their PMIS's or certain key 
applications were not operational over a sufficient period of time.* Port- 

site visits andT system was not yet operational** at the time of the 

nnt ^ Boston's statistical and management reporting capability was 
not yet operational. In these two sites, the feasibility study focused ^on the 

changefprasSctlJal"^ baaeUne on criminal juatica parformanca and following 

t^conatruJ? f d^jHa^fs^pannlng^al Sal 1=^ ^o^nlCtlng^S? 

“ther partinar.t performance mea- 

sures Identified in the evaluation framework. It was also desired to include 
data on control variables, such as caseload and type of case. The field 

examination of alternative methods of data collec- 
Hro ’ dsta from processing of computer files, tabulations 

tabul’aMnn f previously produced monthly statistical reports, and 

tabulations drawn from a manually retrieved sampling of court case files. 

* Operational two or more years. 

**For the purpose of this study, a PMIS was considered operational if data 
were being entered and if outputs (e.g., visual displays in response to 
inquiries and/or printed reports) were being produced. 






Vk, • 4 . ^ consisting of 36 monthly summaries was constructed for 

the city of Norfolk, starting with January 1977, or six months after the PMIS 
became operational. The following performance measures were computed for 
felony 1, felony 2, and other felony cases; 

• number and rates of cases terminated, by plea; 

' • number and rates of cases terminated, by court dismissal; 

• number and rates of cases terminated, by nolle by prosecutor; 

• number and rates of cases terminated, by trial; 

^ • niean days from arrest to indictment; 

• mean days from indictment to trial; 

i| mean days from trial to sentencing; 

^ • number of fugitives added; 

• number of fugitives apprehended; and 

• cases set for trial. 

^ 4 -h u certain workload variables were available, including 

the number of cases commenced: type, number of hearings, and jury-trial days 
A measure of court backlog was available through a spLlal manual system ^ 

maintained by the court administrator, but was not kept in the computer 
syst©ni* 

woo r .1 4 - 4 - order to discern patterns of change, each performance measure 
, was plotted against time in producing a two dimensional graph. To facilitate 

visual comparisons of the relative magnitude of change across the va^'ious 

vertical scale (i.e., the impact measures) were 
standardized to vary from -2 to + 3 , with some outliers. 

^ 4 - 4 ,- DMTc- performance measures exhibited a change in pattern followino 

this PMIS implementation. The number of felony 1 and 2 cases terminated by 

between January, 1977 (time period 1 in the^ 
1978 (time period 15); then appeared to start increasing in a 
parabolic fashion as depicted in Figure 7. A possibly related pattern of 
change was observed for mean days from indictment to trial; this pattern 

a leveling off.around the 15th month, as shown in 
Figure 8. The third variable for which a change in pattern occurred also 
would appear to be related. A rising pattern in the number of felonies dis- 
aboufthB other than felony 1 and felony 2, was observed starting 

r figure 9). These three patterns could be consistent 

w h a policy of reducing backlong and delay of serious cases through more 
intensive plea bargaining and court dismissals. This hypothesis Was examined 
during the impact analysis described below. » 
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Months 

Figure 7. Cases terminated by guilty plea, felony 1<&2 




Duration 

(Days) 




1 3 57 9 11 13 15 17 19 21 23 25 27 29 31 33 35 



Months 



Figure 8. Mean days, indictment to trial 



Standardized 
Number q, 5 
of 
Cases 




Months 



Figure 9. Other felony cases dismissed by court 
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It was desirable to attempt to adjust performance measures to 
indicate relative efficiencies, that is to divide by caseload volume. We 
would like to have divided each month's figures by the backlog (defined as 
cases pending for prior month, plus cases commenced, less cases terminated). 
However, because the backlog of pending cases was not available in the 
computer system, a surrogate measure, cases commenced, was used to adjust 
for workload. 



The analyses indicated no substantial change in -patterns of per-- 
formance oyer the 36 month period for many of the measures, whether or not 
account was taken of the court's workload, as indicated by cases commenced. 
For example. Figure 10 depicts the pattern observed for the proportion of 
felony cases ^terminated by trial. 

Additional data for measuring performance are maintained on 
Norfolk's TRACER system, but these data were available for only a portion of 
the 36 month time series period. These data include; 

• Trial outcomes — guilty, not guilty 

• Convictions — on original charge, or reduced charge 

• Cases terminated — filed this term, filed five months or less 




The measurement of the impact of the PMIS on the prosecution/court 
process involves two aspects of analysis — descriptive and explanatory. 
First, changes in patterns of prosecution/court performance need to be 
detected. Second, the analysis must attempt to determine whether the change 
can be attributed to the PMIS. The latter explanatory analysis attempts to 
rule out effects of external events such as policy or personnel changes. 



Each variable was plotted over the 36 month time frame. The pattern 
of variation in several of the performance measures exhibited a change around 
March, 1978, the 15th month. Thus, a change in performance was observed. 

This raised the tjuestion — did the change arise due to the use of TRACER? As 
indicated in the discussion on the performance measures, the observed changes 
could have been due to the change in chief judge and the institution of a 
master calendaring procedure for assigning judges to cases. According to the 
clerk of the court, it also could have been partially due to the improved 
utilization of TRACER. This finding would be consistent with a hypothesis 
that after a 15-month "break-in" period TRACER assisted the prosecutor and 
court in monitoring cases more efficiently; thereby increasing pleas and 
reducing time delays. Insufficient information was available to decide this 
aspect of the analysis. 



In order to identify and measure any impacts of the PMIS, it is use- 
ful to hypothesize a set of expected impacts. Such hypotheses were formulated 
for Norfolk's TRACER system as part of the Evaluation Framework. Of course, 
the hypotheses actually tested were limited by the availability of data. 
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One hypothesis was discussed above, i.e., that improved case 
tracking added by TRACER enabled the increase in guilty pleas for felony 1 
and 2 cases and helped the court reduce time delays. In order to further 
examine possible impact, a productivity index was hypothesized. This index 
was structured to combine measures of efficiency and /iLprp 

hypothesized that productivity would rise as the impact of the PMIS URALLR, 
in this case) was felt. The productivity index was assumed to vary directly 
with cases terminated by plea, cases terminated by trial, cases set for trial, 
and fugitives apprehended. It was assumed to vary inversely with mean days 
from arrest to trial, cases nolled and dismissed, and fugitives added. These 
assumptions were tested for each class of felonies by means of factor analysis. 
Following validation of these assumptions, the factor loadings were used to 
construct the productivity index. The index, in turn, was used to test the 
hypothesis that the PMIS improved productivity. 

Seven variables were selected for inclusion in the factor analysis, 
consistent with the hypotheses to be tested, the availability of data, and 
examinations of preliminary data plots. The seven variables were total cases 
terminated by plea, by dismissal and by trial, fugitives added, fugitives 
apprehended, cases set for trial, and mean time from arrest to trial. T e 
factor analysis had the effect of reducing the number of variables from seven 

to three. 

The three factors are summarized below, indicating those variables 
which are highly correlated (.7 or higher factor loading); 

(1) Factor 1 (positive productivity) 

Cases terminated by plea - factor loading .887; 

Cases terminated by trial - factor loading .768; 

Cases set for trial - factor loading .744; 

(2) Factor 2 (negative productivity) 

Cases terminated by dismissal/nolle - factor loading .696; 

Fugitives added - factor loading .814; 

- Fugitives apprehended - factor loading .872, and 

(3) Factor 3 (delay) 

- Mean days from arrest to trial, factor loading .949. 

Plots of each factor against time were then performed to attempt to 
detect any changes in patterns, especially in testing the hypothesized impact 
relationships. 
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The fluctuations in Factor 1 identified with positive productivity 
showed very little change in trend over time. A similar lack of any idc»nti- 
fiable impact of the PMIS (see Figure 11) resulted when the variables were 
converted to rates, i.e., divided by the corresponding figures for cases com- 
menced. The fluctuations in Factor 2 associated with negative productivity 
variables also appeared to be random either in unadjusted form (see Figure 12) 
or when adjusted for workload. The delay factor also showed no substantial 
change in pattern. Thus, there was no support for any hypothesized impact 
relationships on the total caseload. However, it remained to be tested 
whether impacts were masked by lumping all felonies together. 

A finding that there was little impact of TRACER on the court and 
prosecution system would have been consistent with certain observations during 
the site visits. TRACER was being used by the prosecutor for case and defend- 
ant status monitoring via terminal inquiry. However, based on interviews of 
the prosecutor's office in Norfolk, it was concluded that TRACER was under- 
utilized by the prosecutors. The court and police appeared to understand and 
use TRACER capabilities, but the prosecutor's office needed to be educated 
and upgraded especially in the use of management statistics. 

The above factor analysis showing no discernible impact occurred 
using variables that measure "all felonies" as one class. When classes of 
felonies were looked at separately, some changes were observed, as discussed 
in the performance section. The observed changes in pattern of felony 1 and 
2 pleas, average days from indictment to trial and court dismissal rates lent 
some support to a theory that TRACER was having some impact by helping to 
expedite case processing. A followup phone call to the Norfolk Circuit Court 
revealed that the court started to use TRACER outputs, during the observed 
time period, to examine the age of cases and as an aid in enforcing stricter 
controls on case processing. Emphasis was placed on the most serious charges 
in indictments by seeking guilty pleas in favor of dismissing lesser included 
charges. The court believed that as a direct result, the number of felony 1 
and 2 pleas increased, the rate of dismissals increased, and the average days 
from indictment to trial decreased. The data appeared to support the court's 
belief that TRACER had such an impact. The project staff also learned that a 
new chief judge took over about the time that the change was observed. He 
had a reputation for being tough and instituted a master calendaring procedure 
to preclude judge shopping, i.e., a judge was not assigned to a case until 
the day of trial. This change might also help to explain the observed change 
in disposition rates. 

A more conclusive time series analysis would be possible through 
the use of cross-sectional data involving a group of jurisdictions. This type 
of design would enable the inclusion of additional control variables. 
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D. Conclusions Regarding Feasibility Issues 





1 . Cost-Benefit Analysis 

Although several cost-benefit models exist and these models have 
been used for predictions (predominantly PROMIS sites), no evidence was 
found among the 19 jurisdictions visited during this entire study that retro- 
spective cost-benefit analyses have been performed after PMIS implementation. 
Analysis after PMIS implementation will require close cooperation among the 
PMIS users, particularly to obtain agreement that users are willing to provide 
value judgments (i.e. , give dollar values) for intangible benefits. 

The goals that a prosecutor establishes for his PMIS are helpful 
in identifying PMIS benefits. During this study, prosecutors were asked to 
identify their PMIS goals, and to rate the PMIS regarding "expected" and 
"actual" contributions toward goal attainments. The ratings were given on a 
scale of 1-100, but the same approach can be helpful in assigning a dollar 
value to related benefits. 

Many PMIS projects are implemented in phases. The most common 
practice is to start by entering data for one type of case (e.g. , felonies). 
Once satisfied with the data entry and basic day-to-day operations of the 
system for the initial case type, the next type of case (e.g., misdemeanors) 
will be initiated. PMlS applications may also be implemented in phases. The 
trend is for daily outputs, such as calendars and schedules, to be initiated 
first, followed sometime later (usually months later) by management reports, 
such as statistical outputs. PMIS support to daily prosecution operations 
will reflect some benefits. If cost-benefit analysis is to be used in con- 
junction with the analysis of PMIS impact on the prosecution process, such an 
analysis should not be attempted until after management outputs have been 
produced for 9-12 months Uhich will also allow time for a "learning curve" 
regarding use of such outputs). 

Performing retrospective PMIS cost-benefit analysis is considered 
feasible provided that the PMIS is operational for 24-26 months prior to 
analysis. 

2. Availability of Baseline Data for A Priori Impact Analysis 

A feasible approach toward construction of the baseline is to com- 
bine the use of several data sources, i.e., available statistical reports and 
special tabulations drawn from a sampling of case records, to generate per- 
formance measures. 

Where reports on caseload and dispositions are available, they can 
provide a point of departure in constructing the baseline. Comparability and 
reliability of annual reports are open to question, and some effort needs to , 
be made to check validity through an external source. A statistical sample 
of several hundred cases should;, be drawn manually from the files of cases 
terminated during the baseline year. Most data could be obtained from court 
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files. Some data could be obtained from the prosecutor's files. Experience 
in the field investigations indicated that it would be feasible to construct 
such a data base for computation of the required performance measures if 
staff time of about 20 minutes per case could be allocated to the data collec- 
tion effort. 




During the field investigations, the manually maintained prosecution 
and court case files were examined to ascertain the feasibility of extracting 
data for constructing a time series. Most jurisdictions maintain their case 
files in sequence by court case number, and generally these numbers are 
assigned chronologically on the date the case is filed in court, i.e., accepted 
for prosecution. In order to obtain statistics on cases for which prosecution 
is declined, one would have to search the prosecutor's files. Such cases are 
usually filed by defendant's name or police complaint number. Thus, the most 
convenient method of selecting the sample is based on the date of case filing. 
If a large enough sample is drawn, starting with cases filed about one year 
before installation of the computer (to allow time for building up case termi- 
nations), then a representative time series could be constructed. The sample 
would need to be large enough to generate a sufficient sampling of all the 
desired subpopulations, e.g. , cases declined, cases going to trial, cases 
dismissed, and other categories for v^hich separate performance measures are 
desired. The size of the sample required will also depend on the purpose of 
the task. For example, to produce a set of baseline statistics for a single 
time frame, a sample of about 400 cases should be sufficient. However, to 
generate the complete time series data base for impact evaluation would 
require about 24 monthly summaries. To construct such a data base would 
probably require a sample exceeding 1,200 cases to be followed through to 
disposition. Using an estimated 20 minutes per file, about 400 person hours 
would be required to tabulate the data. The latter type of effort is con- 
sidered neither feasible nor necessary by the project staff. Since the 
purpose of the task is to evaluate the impact of the PMIS, it is reasonable 
to assume that the PMIS would be available to monitor its own performance, 
and require manual samplings only as supplemental data. 

The feasibility of collecting time series data from computer based 
records was investigated by obtaining computer tapes from Norfolk, Golden and 
Oakland. The Oakland tape was not readable on Westat's IBM 370 due to some 
hardware incompatibilities. Oklahoma City could have provided the tape, but 
required that a written request be submitted to the Governor's Commission for 
approval. This was not feasible, in view of some potential political sensi- 
tivities to release of such data and because of time constraints of the 
project. The Norfolk (IBM) and Golden (Honeywell) tapes were readable. The 
project budget did not permit the use of the Golden tapes in time series 
analysis. However, based on use of the Norfolk tapes, it was the judgment of 
the staff that it would be feasible to construct a time series data base from 
the Golden tapes if more time and resources were made available. 
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Analytical Techniques 

Analytical techniques were examined for feasibility in three areas: 

• cost analysis; 

• psrformance measurement; and 
e impact evaluation. 

structlng an 

ppropriate and comparable measures of cost and performance and nnllpof fh 
required data over the relevant li„e periods are’^dLcS^sr^lslere in iS, 
Nowever, given uniformly defined data elements, the analytical tech 

pe?formance^?or“;L^o??:^Srrf^cTtuTS 

are more chSlengin^'w 

stCdy“ considered outside the scope of this feasibility 

rni feasibility issue examined, with respect to analytical tech 

thrPMiq''°"‘'fh" adequacy of methods of evaluating the actual impact of 

the PMIS on the court environment. In accordance with the interruoJed timf 
series design presented earlier, a data base of monthly actlvityrKSn^m at 
least two years of criminal justice operations, was desired preferabl^ ^ ^ 

Given such considered operatiLal. 
Given such a data base, techniques were needed to (1) detect any chanoes in 

extenf trilhich justice system activity, and (2) determine the 

extent to which those changes are attributable to the Intervention of the 



niques'?' Se ^eL^bUUy'^o'rappJylng Jllterr^p'Jed L'mf s^rie1'L:ly\L'\'ech- 

assessmentsTynfo^^:^t^ inttSw^1rolt.e 

activity 
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this task is data base development Fnr » ^ i ^ crucial aspect of 
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IV. SUMMARY OF FINDINGS 



A. Introduction 

Issues emerging from this study have been discussed throughout the 
report. These discussions include findings that are based on analysis of the 
collected data, a literature search, opinions of PMIS users and planners, and 
judgmental assessments made by the study team. 

The purpose of this chapter is to summarize the findings of major impor- 
tance. References to the sections of this report where discussions of the 
findings can be found are shown parenthetically. 



B. Nature and Scope of PMIS Use 

Automated PMIS projects, are being used or planned primarily by larger 
(25 or more employees) prosecution offices, (Section II D.1). More PROMIS 
( projects (as a group) are in use or being planned than any other single 

project, (Sections I B and II D.1). 

Types of PMIS projects in use include: systems dedicated to and con- 

trolled by the prosecutor; systems controlled by one agency (prosecutor or 
court, for example) and shared by the prosecutor and courts; integrated 
( systems, controlled by a central data processing facility and shared by two 

or more user agencies; and, integrated systems, controlled by a central data 
processing facility and shared by multiple jurisdictions, as well as multiple 
agencies. PROMIS, as well as non-PROMIS projects, are employed in all of 
these types of systems, (Section I B). 

C The most common and most effective type of PMIS is the integrated system. 

Advantages of the integrated system include: avoiding duplication of hard- 

ware, staffing and work effort; better data quality; and, greater depth in 
data processing expertise. Greater interagency cooperation and a higher 
degree of user satisfaction are found more frequently in integrated systems 
than in dedicated projects; and, confidentiality safeguards in the observed 
^ integrated systems were satisfactory (Sections II E.1 and II E.3.c). 



C. The State-of-the-Art in PMIS Use 

Assessments made by the project staff, during the first segment on-site 
surveys, indicated that many PMIS's were used effectively to support day-to- 
day case processing operations, but few were used effectively as strategic 
management tools, i.e., to support management policy development and decision- 
making. (For five of seven features supporting day-to-day operations, more 
than 90 percent of the PMIS projects had some capability and more than 80 
percent had state-of-the-art capability. The same was true for only one of 
nine management features (Sections II E.3.b and II E 3.c). Nothing was 
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D. PMIS Transfer Potential 
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(e.g., San Bernardino's interface control programs - Section II E 1 
offer the potential for transfer of non^PROMlS oroiprtr i u ■ 

applications within those projects. ^ projects, or at least certain 

versions'^ofVROMTS^nnnf^’^fi*^^^^ been well documented and the latest 
several verarons of PROMIS have been deelgn^ed as a 
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new capabilities and features being added at each stage of development. The 
MINI and MAXI PROMIS tailoring features and their capability to run on 
various hardware configurations offer prosecutors flexibility in adapting the 
system to meet local requirements and in selecting the type of computer system 
for thcix project (Section I B). 

Tailoring and self-documenting features of the MINI and MAXI PROMIS 
versions (Section I B) should help reduce transfer costs; however, experience 
with PROMIS tailoring had not progressed far enough at the time of the study 
to make this determination (Section II E 4.c). 



E. Funding of PI* 'IS Projects 

f 

PMIS projects, both PROMIS and non-PROMIS, have relied heavily on Federal 
funds and outside consultants to develop or transfer their systems (Section 
II E 2.c). Lack of money and people is the main reason, cited by both larger 
and smaller prosecutor's offices, for not using or planning a PMIS (Section 
II E 2.d). 

Federal funding policies (e.g., LEAA's Incentive Fund Program) encourage 
PMIS transfer (Section II E 2.b), as indicated by the number of oper3- 
tional, planned, and in-transfer PROMIS projects discussed above. Because of 
the extensive funding of PROMIS, LEAA asked that recommendations be developed 
as to where future PMIS funds should be concentrated (e.g., further PROMIS 
( development, technical assistance for PROMIS transfers, or development of non- 

PROMIS projects). Based on the findings summarized above, it is recommended 
that future Federal funds be allocated in the following manner: 

s First priority - technic al assistance to PROMIS transfer projects , 
particularTy~where experienced data processing personnel are not 
( readily available to cope with PROMIS implementation. In any case, 

consultants or contractors who have had experience in transferring 
PROMIS to one jurisdiction will be able to provide valuable 
assistance to other jurisdictions. If maximum benefits are to be 
gained from the heavy investment in PROMIS, effective implementa- 
tion of the systems should be the primary concern to LEAA. 

Second priority — devel opment of non-PROMIS projects . Although the 
latest PROMIS versions offer prosecutors flexibility in system 
design, there may be valid reasons for selecting a non-PROMIS 
approach. For example, it may be more economical or more desirable 
to expand an existing criminal justice information system (e.g., a 
court or police system) to incorporate the prosecutor's require- 
ments, rather than attempting to interface PROMIS with the existing 
system or rather than implementing a separate system just for the 
prosecutor. Effective PMIS projects have been developed in this 
manner, such as the TRACER system in Norfolk, VA, which started as 
a police system. 
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System modifications will be necessary in every transfer situation. 
PROMIS tailoring features may help simplify such modifications, but 
many data processors believe that the disadvantages of transfer 
(Section II E 2.b) outweigh the advantages; and, many of them hold 
to the premise that a PMIS should be designed to fit the specific 
procedures of the jurisdiction and that the jurisdiction should not 
have to alter procedures to fit a PMIS.* Efficiency of operations 
is another factor that may prompt modifications. In Golden, Colorado, 
for example, consideration was being given to modifying new PROMIS 
to utilize the central computer's own operating system rather than 
the one contained in the PROMIS package. 

Although most non-PROMIS systems may not be documented well enough 
to support system transfer (Section II E 2.e), innovative approaches, 
such as interface control programs developed in San Bernardino 
(Section II E 2.b), do offer the potential for transfer. PROMIS, 
therefore, is not tht only alternative to PMIS support, and prosecu- 
tors should not be discouraged from examining those alternatives by 
denying them Federal funds for that purpose. The Justice Department 
should encourage innovation and non-PROMIS projects have demonstrated 
techniques that compare favorably with PROMIS projects. 

Third priority - further PROMIS development . PROMIS has been in 
various stages of development since 1971. Each version of PROMIS, 
emerging from this developmental effort, has provided improvements 
over previous versions, with the latest one (MAXI PROMIS) offering 
the greatest potential for transfer because of increased flexibil- 
ities. Some jurisdictions have delayed PROMIS implementation 
pending the availability of the latest version.** Other jurisdic- 
tions, operating some version of PROMIS, are planning to change to 
MAXI PROMIS (Section II E 4.c). Among jurisdictions planning to 
change to MAXI PROMIS and among those planning a PMIS, but who have 
not decided on what system to acquire, PMIS planners are anxious to 
see the latest version in operation in order to assess its capabil- 
ities and to determine the degree of difficulty in implementing 
MAXI PROMIS.** Users and developers of automated projects are 
seldom completely satisfied with the operations of their system; 
they v;ill (and should) continuously make improvements. At some 
point, however, priorities should be shifted from system development 
to system operations in order to gain maximum benefit from the 
developmental work already accomplished. 

Assigning priorities in this manner would have the effect of first 
capturing a return on Federal investments already made in repro- 
gramming and enhancing PROMIS for transfer; secondly, capturing 



* Based on interviews during on-site surveys. 

**Based on comment received from PMIS planners during Westat telephone survey 
of jurisdictions operating, transferring or planning PROMIS. 
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benefits on investments made by local jurisdictions in criminal 
justice systems already in operation; and thirdly, allowing prose- 
cutors to gain more experience using existing applications before 
investing resources in new enhancements. 
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F. . Availability of Evaluation Data 

In general, data needed to evaluate PMIS projects are available; and, it 
is feasible to collect the data in varying degrees of detail and from various 
sources (Section III B). 

• Caseload data needed for time series analysis is available, but the 

source for those data will vary. Sources include: PMIS files 

(e.g., history tapes); monthly statistical reports (either PMIS or 
manually produced); and/or, case files from which data can be manual 
extracted (Section III B.1). 

• Data concerning total PMIS developmental and operational costs can 
be obtained, but will vary in the detailed backup data leading to 
those totals (Section III B.2). 

■ PMIS benefits are relatively easy to identify, but many benefits are 
difficult to quantify because they require value judgments by the 
PMIS users (Sections III B.1 and III C.1). 



G. Feasibility of PMIS Evaluations 
1 . Cost -benefit Analysis 

Criminal justice agencies, other than the prosecutor, benefit from 
PMIS operations. All agencies with direct access to the system, or which 
receive PMIS outputs indirectly, should be included in any PMIS cost-benefit 
analysis (Section III C.1). 

Cost-benefit analysis should be conducted only after the PMIS has 
been in operation long enough (24-26 months) to stabilize (i.e., planned 
dual operations, PMIS and manual functions, have been discontinued; all 
phases of PMIS implementation have been completed; and, users have reached 
the apex of the learning curve) (Section III D.1). 

If a jurisdiction has made cost and benefit predictions (pre-PMIS 
implementation), the cost-benefit model used for those predictions should 
also be used when conducting the cost-benefit analysis (post-PMIS implementa- 
tion) (Section III B.1). 

Prior to conducting cost-benefit analysis, PMIS users within the 
jurisdiction must agree that they are willing to provide dollar values for 
intangible benefits (Section III D.1). 









PMIS 

2. Performance Measures 

rates of ter^l^atadlSs^^Jun^ <' 9'. "-bers and 

arrest through intermediate court actions to”^aentr=°'^^^°"°’ 

measures) (Section III C.2). ntencing, and various workload 
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requires the detection of changes in dLsp Prosecution/court process 
and a determination of whether such P ®'^‘^^^°"/court performance over time 
(Section III C.3). such changes can be attributed to the PMIS 
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H. PMIS Institutionalization 
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Since Federal funding and outside technical assistance have been so 
essential to PMIS implementation thus far (Sections II E.2.c and II E.2.d), 

PMIS implementations can be expected to continue in the future only if such 
support is continued. 

In jurisdictions where the PMIS is operated by a centralized data pro- 
cessing facility, retaining qualified personnel has been a problem because the 
( pay scale for data processors is usually lower in the local government than in 

the private sector.* In jurisdictions where the PMIS is operated by the prose- 
cutor, personnel in the prosecutor's office have been trained to operate the 
PMIS (as noted in three out of four prosecutor operated systems surveyed); 
these personnel are also looking forward to transferring their newly acquired 
skills to better paying jobs.* Between the two situations, the centralized 
C facility offers the best opportunity for the prosecutor to obtain access to 

and retain qualified data processing personnel to support PMIS operations, 
because of the large personnel staff (Section I B) usually associated with 
such a facility. 

Local funds are currently used to sustain PMIS operations in those juris- 
C dictions where Federal funding has expired. It is only reasonable, however, 

to expect local budgetary decisionmakers to insist on some evidence that PMIS 
benefits and/or impacts on the judicial system are worth their cost before 
increasingly scarce funds are allocated to continue PMIS operations. None of 
the jurisdictions surveyed had performed a retrospective cost-benefit analysis 
based on actual experience. However,, cost-benefit predictions had been made 
C in some cases (Section III B.3). Although PMIS benefits have not been quanti- 

fied, top prosecution managers remain enthusiastic about their PMIS capabil- 
ities (Section III C.1), particularly where they have been directly involved 
in PMIS development and display a "pride in ownership" (Section II E.2.b). 

In these cases, user enthusiasm may offset the lack of documented evidence 
of PMIS worth, to justify funds for PMIS operations. In other cases, where 
C prosecutor's expectations are not being met by the PMIS (Section II E.3.a), 

less support for continued PMIS operations can be expected. Therefore, 
in-depth PMIS evaluations need to be performed to provide users with suffi- 
cient justification for continued funding of their PMIS. 

In the opinion of the study team, PMIS utilization will continue to 
C expand in the future provided that Federal funding is available to stimulate 

such projects. Unless the value of PMIS projects can be sufficiently demon- 
strated through independent evaluations, the availability of Federal funding 
cia'ri be expected to decline. It is also expected that iitate and local govern- 
ment funding of PMIS projects will not pick up all the slack, unless cose 
effectiveness can be demonstrated through credible evaluation processes. 

C 



*Based on interviews with data processing personnel during site visits. 

€ 



109 




c 



Greenwood, G.M., et al. Court’s Equipment Analysis Project - Data 

Processing and the Courts - Reference Manual, National Center for State 
Courts, September 1977. 

Senn, 3. A. Information Systems in Management , Wadsworth Publishing 
Company, Inc., Belmont, California, 1978. 

Institute for Law and Social Research (INSLAW), "MINI PROMIS Documenta- 
tion Volume 1: Technical System Description," Draft P/78. 

INSLAW, PROMIS Newsletter, December 1979. 

Oacoby, 3. "The Prosecutor's Charging Decision: A Policy Perspective," 

Law Enforcement Assistance Administration, 3anuary 1977:16-19. 

Byrne, G. j; et al. Automated Case Management in, the Prosecutor's Office . 
Chicago: National District Attorney's Association, 1977. 

Polansky, L.P. Computer Use in the Courts: Planning, Procurement and 

Implementation Considerations . Washington, D.C.: Criminal Courts Tech- 

nical Assistance Project, American University Law School, 3une 1978. 

Search Group, Inc. State Oudicial Information Systems - State of the 
Art . Technical Memo No. 11. Sacramento, California, 3une 1975. 

National Advisory Commission on Criminal 3ustice Standards and Goals. 
Criminal 3ustice System , 1973. 

National Center for State Court, National Criminal 3ustice Information 
and Statistics Service, LEAA. "State Court Caseload Statistics: The 

State of the Art." August 1978. 

INSLAW. An Overview of Minicomputer PROMIS . Washington, D.C. 1978. 

Symposium on Criminal 3ustice Information and Statistics Systems. 
"Courtran: A Modular Management Information and Research System for 

the Courts." Washington, D.C.: Project Search, LEAA, October 1972. 

Search Group, Inc. Minicomputers and Criminal 3ustice - Introducing 
a New Technology . Sacramento, California: Search Group, Inc, , 

Technical Report No; 23, 1978. 

INSLAW. "Transferability." Washington, D.C.: PROMIS Briefing, 

Report 20, 1977. 

Wallace, G.K., and Wilson, 3. "Oklahoma's D.A. Reporting System Keeps 
Tab on Career Criminals." Oklahoma Crime Commission, APCD Bulletin, 
3uly 1978. 



O 




16. INSLAW. PROMIS Newsletter, May 1979. 

17. Kleps, R.N., and McKay, M.B. "Conceptual Design for Court Information 

Systems - The Integrated Court/Automation Information System (ICAIS)." 
Proceedings of the Second International Symposium on Criminal 3ustice 
Information and Statistics Systems, April 30 - May 2, 1974. 

18. Lyman, T.R. "Performance Evaluation of a Criminal 3ustice Information 

System: A Transferable Methodology." Urban Systems 3ournal . New York: 

Pergamon Press, 1977. Volume 2, Nos. 2-3, 63-77. 

19. Project Search Committee on Security and Privacy. Security and Privacy 

Publications . California: Project Search, May 1973. 

20. "Federal Rules and Regulations." Federal Register: Title 28 - 

Chapter 1, part 20.3(b). March 19, 1976. 

21. National Criminal 3ustice Information and Statistical Service, LEAA. 

Privacy and Security of Criminal History Information, An Analysis of 
Privacy Issues , 1978. 

22. Greenwood, 3.M., et al. "Court Equiment Analysis Project - Data Pro- 

cessing and the Court Guide for Court Managers." National Center for 
State Courts, September 1977. 

23. Memo from 3ames Devine. "Funding the Development of Computerized 

Criminal 3ustice Information System." LEAA, May 4, 1972. 



Wildhorn, S., et al. , Rand Corporation. Indicators of 3ustice; 

Measuring the Performance of Prosecution, Defense and Court Agencies 
Involved in Felony Proceedings - A Guide to Practitioners . National 
Institute of Law Enforcement and Criminal 3ustice, LEAA, May 1977. 



25. INSLAW. Research Uses of PROMIS Data. INSLAW Briefing Paper No. 10. 
March 1977. 



26. Church, T., Carson, A., Lee, -3., and Tan, T. 3ustice Delayed - Thr3 

Pace of Litigation in Urban Trial Courts, The National Center for 
State Courts, 1978. 

27. Brosi, K.A. Cross-City Comparison of Felony Case Processing. 

Washington, D.C.: INSLAW, April 1977. 

28. INSLAW. "Analysis of Costs and Benefits." Briefing Paper No. 19. 

3anuary 1977. 

29. Kriendel, B., et al. National Evaluation Program, Phase I, Final 

Report; Court Information Systems, Preliminary Findings and Issues. 
MITRE Corporation, 1975. 





( 



30. Weidman, D.R., et al. "Intensive Evaluation for Criminal Justice 

' Plannang Agencies." LEAA, July 1975. 

31. Nay, J.N., et al. Working Paper 5070-3, Work Description for a Phase I 

Study. The Urban Institute, February 1977. 

32. Jacob, H., and Eisenstein, J. "Felony Justice," Little, Brown & Associ- 

' ates, Boston, Massachusetts, 1978. 

33. National Institute of Law Enforcement and Criminal Justice (LEAA), 

j^ntensive Evalu ation for Criminal Justice Planning Agencies . July 1975. 



t 



( 



€ 



C 



C 



c 



c 



c 



A-4 







Adams, Eldridge. Courts and Computers . Chicago, Illinois. The American 
Judicature Society, Library of Congress, 1972. 

American Bar Association Commission on Standards of Judicial Administration. 
Trial Courts, Standards Relating To , Chicago, Illinois, American Bar Associ- 
ation, 1975. 

Association of Trial Attorneys. Automated Court Case Management in the Prose- 
cutor's Office . 

Blain;^,, Gerald. "Computer-Based Information Systems Can Help Solve Urban 
Court Problems," Judicature , November 1970. 

Blumstein, A., and Larson, R. "Models of a Total Criminal Justice System." 
Operations Research . March-April 1969. 

Bockelman, Melvin F. ALERT II - Progress Toward A Computerized Criminal 
Justice System. National Symposium on Criminal Justice Systems. October 
1972. 

Brounstein, Sidney H. "The Potential Impact of Information Systems In Evalu- 
ating Prosecution and Court Policy." Proceedings of the First Annual 
National Evaluation Conference LEAA, Washington, D.C,; February 1977. 

Burroughs. PROMIS/JUSTIS, Prosecutor's Management Information System . 
February 1977. 

Church, T., Carson, A., Lee, Jr., and Tan, T. Justice Delayed - The Pace of 
Litigation in Urban Trial Courts, The National Center for State Courts, 

1978. 



Ebersole, Joseph L., and Hall, James A. Jr. "Courtran; A Modular Manage- 
ment Information and Research System for Courts." Proceedings of the 
International Symposium on Criminal Justice Information and Statistics 
Systems . Washington, D.C., Project Search, LEAA, October 1972. 

Friesen, E.C., Gallas, N.M., and Gallas, E.C. Managing The Courts. Bobbs- 
Merrill Co. , 1971. 

Greenwood, Peter. Potential Uses of The Computer in Criminal Courts . 
(P-4581) Rand Corporation. February 1971. 

Hann, Robert G. Decision Making in the Canadian Criminal Court System; 

A Systems Analysis. Center of Criminology, University of Toronto. 

July 1973. 



B-2 





INSLAW - Prosecutor's Management Information System (PROMIS) Manual 

Vol. I - System Overview 

Vol. II - System and Program Description 

Vol. Ill - Data Preparation Guide 

Vol. IV - Software Implementation Guide 



INSLAW Special Reports 

1. Management Report Package for PROMIS , Oct. 1976 

2. Generalized Inquiry Package for PROMIS , Oct. 1976 

The Pocket PROMIS - A Basic Guide to the System and Its Use, Oct. 1976 
(for D.C.) 



The Pocket PROMIS - A Basic Guide to the System and Its Use , Oan. 1978 
(for N.Y. ) 



ISLAW PROMIS Research Project 
Publication 



Title 



(Draft) 



Highlights of Interim Findings and Implications , 1977 

Expanding the Perspective of Crime Data; Performance 
Performance Implications for Policymakers , 1977 

Curbing the Repeat Offender ; A Strategy for Prose- 
cutors , 1977 

What Happens After Arrest , Aug. 1977 

The Prosecution of Sexual Assaults , Nov. 1978 
(Executive Summary) 

Does the Weapon Matter , no date (Executive Summary) 

The Role of the Victim in the Prosecution of Violent 
Crimes , May 1978 (Executive Summary) 

Plea Bargaining; Who Gains? Who Loses? , no date 
(Executive Summary) 

Pretrial Release and Misconduct in the District of 
Columbia , Oct. 1978 (Executive Summary) 

The Deterrent Effect of Capital Punishment; A Cross- 
State Analysis of the 1960's, no date 



B-4 



Publication 

(Draft) 



(Draft) A Descriptive Profile of Case Processing Time in the 
district of Columbia Superior Court , June 1976 

(Draft) A Description of Patterns of Recidivism in the Distric t 
of Columbia , Nov. 1976 

Progress Report, PROMIS Transferability Contract, 1-31 
Oct. 1978 

INSLAW - PROMIS Newsletters 

Vol. 1, Number 2, Sept. 1976 
Vol. 2, Number 2, Dec. 1977 
Vol. 3, Number 1, Apr. 1978 
Vol. 3, Number 2, Oct. 1978 

Accelerating Reform in the Courts Through Technology Transfer. Oune 1975 
(presented to NATO) ^ ‘ 

Guide to Court Scheduling - A Framework for Criminal and Civil Courts , 1976 

(Draft) An Overview of MINICOMPUTER PROMIS , 10/78 

(Draft) MINI PROMIS Documentation, Volume I: Technical System Description, 

10/78 ^ ^ ’ 

PROMIS Research Project, Research Design Report, End of First Year 
Dec. 1975 

PROMIS for the Nonautomated or Semiautomated Office , April 1976 

Cost/Benefit Analysis Reports (Reports cover 71 sites including the 20 PROMIS 
sites which are currently operational) 

Jacoby, Joan E. The Prosecutor's Changing Decision; A Policy Perspective . 
U.S. Department of Justice, Law Enforcement Assistance Administration, 
National Institute of Law Enforcement and Criminal Justice. January 1977. 

Jennings, John B. Final Evaluation of Manhattan Criminal Court's Master 
Calendar Project (R-1285-NYC) . New York City, Rand Institute. November 
1973. 



Judicial Data Center, PROMIS Feasibility Study, State of Michigan , Detroit, 
Michigan, July, 1978. 

Kramer, K.Lu, Lucas, H.C. Jr., Hackathorn, R.D., and Emrey, R.C. Computer 
Utilization in Local Governments; A Critical Review and Synthesis of 
Research . University of California, Irvin. September 1973. 



B-5 



Maltz, Michael D. "Privacy, Criminal Records and Information System." Opera- 
tions Research in Law Enforcement, Justice and Social Security. D.C. Heath 
and Co,, Lexington, Mass. 1976. 

Merrill, Dean C. Accelerating Reform in the Courts Through Technical Transfer . 
Institute for Law and Social Research. June 1975. 

MITRE Corporation. National Evaluation Program, Phase I Final Report; Court 
Information Systems; Preliminary Findings and Issues . November 1975. 

. National Evalu ation Program, Phase I Final Report; Court Information 
Systems; Phase II Study . July 1976. 

. National Evalua tion Program, Phase I Final Report; Court Information 
"Systems; A Single Court Information System Project Evaluation Design . 

August 1976. 

National Advisory Commission on Criminal Justice Standards and Goals. Courts . 
Washington, D.C. U.S. Government Printing Office, January 1973. 

. Criminal Justice Research and Development. 



. Criminal Justice System . January 1973. 

National Advisory Commission on Law Enforcement and Administration of Justice. 
Task Force Report; The Courts, Appendix E . U.S. Government Printing 
Office, 1967. 

National Center for State Courts, Reference Manual. Data Processing and the 
Courts , Publication No. R0033g. September 1977. 

National Criminal Justice Information and Statistics Service. Privacy and 
Security Planning Instructions - Criminal Justice Information Systems' . 

U.S. Department of Justice, LEAA. Revised April 1976. 

National District Attorneys Association. Automated Court Case Management in 
the Prosecutor's Office . 

National Legal Aid and Defender Association, Defender Management Information 
Systems Feasibility Study LEAA, 1979. 

Navarro, Joseph A. and Taylor, Jean G. "An Application of Systems Analysis 
to Aid in the Efficient Administration of Justice." National Institute on 
Computers in Courts , Battle Creek, Michigan. W.K. Kellogg Foundation, 1972. 

Nay, J.N., et al. Working Paper 5070-3, Work Description for a PhaPe I Study. 
The Urban Institute, February 1977. 



( 



B-6 



Office of Criminal Justice Plans and Analysis Information Systems and Statis- 
tics Division. A Quality Control Analysis of Criminal Justice Information 
System Data in the District of Columbia , OBTS Technical Report Number Six. 
September 1977. 



Oklahoma Crime Commission. Material concerning Arrest Disposition Reporting 
System . December 1978. 

. Arrest Disposition Reporting System (APRS) . December 1978. 

Organization for Economic Cooperation and Development. Automated Information 
Management in Public Administration . France, 1973. 

Polansky, Larry P. "Contemporary Automation in the Courts." Law and Computer 
Technology. November 1973. 



. The Computer Use in the Courts; Planning, Procurement and Implementa- 
tion Considerations . American University, June 1978. 

. "Contemporary Automation in the Courts." Proceedings of the Second 

International Symposium on Criminal Justice Information and Statistics 
System . May 1974. 

President's Commission on Law Enforcement and Administration of Justice. 

Task Force Report; The Courts . 1967. 

. The Challenge of Crime in a Free Society . 1967. 

Programming Methods, Inc. Report on the Development of a Criminal Court 
Calendar Scheduling Technigue . New York, N.Y. 

Project SEARCH . Proceedings of the Second International Symposium on Criminal 
Justice Information and Statistics Systems. May 1974. 

The Rand Corporation, Criminal Justice Models; An Overview , April 1976. 

. Indicators of Justice; Measuring the Performance of Prosecution, 

Defense, and Court Agencies Involved in Felony Proceedings - A Guide to 
Practitioners , June 1976. 

SEARCH Group, Inc. SJIS — State Judicial Information Systems State of the 
Art . Sacramento, California 1975. 

. Standards for Security and Privacy of Criminal Justice Information, 

Second Edition ; Technical Report No. 13 (Revised). January 1978. 

. Gavel - A National Model Trial Court Information System Project - 

Technical Report #22. 1978. 



. Listings of Prosecutor's Management Information Systems Currently 

in Use. 



O 



B-7 



Taylor, S., Navarro, J. and Cohen, R. Data Analysis and Simulation of the 
District of Columbia Trial Court System for the Processing of Felony 
Defendants - Research Paper P-415. IDA Science and Technology Division. 
June 1968. 

United Nations Social Defense Research Institute. "Evaluation Research in 
Criminal Justice." Publication No. 11, Rome, N.Y. January 1976. 

. Evaluation Research in Criminal Justice , Publication No. 11, Rome. 

January 1976. 

U.S. Department of Justice (LEAA). National Survey of Court Organization . 
October 1973. 

United States Department of Justice, Washington, D.C., Law Enforcement 
Assistance Administration, National Institute of Law Enforcement and 
Criminal Justice. Report on the National Evaluation Program . June 1975. 

U.S. Department of Justice, Law Enforcement Assistance Administration. 
Quantitative Tools for Criminal Justice Planning . 

U.S. Department of Justice, Law Enforcement Assistance Administration, 

National Institute of Law Enforcement and Criminal Justice. National Eval- 
uation Program Phase I Summary Report: Court Information Systems. March 

1977. 

. Dictionary of Criminal Justice Data Terminology . 1976. 

. State and Local Prosecution and Civil Attorney Systems , March 1978. 

. Offender-based Transaction Statistics; New Directions in Data Collec- 
tion and Reporting . 1975. 

United States Senate, Committee on the District of Columbia. Court Manage- 
ment Study, Part I, Summary. Washington, D.C. U.S. Government Printing 
Office. May 1970. 

Urban Institute. Work Description for a Phase I Study (Working Paper 5070- 
03) . February 1977. 

Weidman, D.R., et al. "Intensive Evaluation for Criminal Justice Planning 
Agencies." LEAA. July 1975. 

Weimer, David Leo. Improving Prosecution; The Inducement and Implementation 
of Innovations for Prosecution Management. 1978. 



Arthur Young & Co. Detailed Design for the Automated Case Following Statis- 
tical Reporting and Calendaring System - Fresno Superior Court . Vols. I- 
III. December 1972. 



C 



B-8 





APPENDIX C. Persons Interviewed at PMIS Sites 



C-1 




( 




Neil Riddle, Data Systems Coordinator 

Robert Johnson, Assistant Director, Bureau of Special Operations 

Florence Linn, Assistant Director, Central Operations 

Michael Genelin, Head, Career Criminal Unit 

Joseph Siler, Special Assistant to District Attorney 

Larry Donoghue, Deputy District Attorney 

Eloise Williams, Data Systems Analyst 
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D. Lowell Jensen, District Attorney 
Don Ingraham, Deputy District Attorney 
Rod Rolefson, Deputy District Attorney 
Kathy Bergland, Deputy District Attorney 
Richard Haugner, Deputy District Attorney 
Ruby Freitas, Administrative Assistant 
Dave Budde, Administrative Assistant 
Peggy Richmon, Records Clerk 
Ninfa Wood, Secretary to District Attorney 
Don Whyte, Deputy District Attorney 
William M. Baldwin, Deputy District Attorney 
Bill McGuinness, Deputy District Attorney 
Yvonne Ayres, DALITE Manager/Programmer 
Diane Bullock, DALITE Operator 
William Kleeman, Deputy District Attorney 

Mike Scanlon, Administrative Office, Public Defender's Office 
Bill Cook, Records Section Director, Hayward Police Department 
C.J. Moret, Chief, Criminal Division, Clerk's Office, Oakland 
Piedmont Municipal Court 

Peggy Hunter, CORPUS Input Section, Alameda County Superior Court 
Dan George, CORPUS Project Manager 

Herbert L. Pike, Office of Court Administrator, Superior Court 
Beverly Graves, Criminal Clerk's Office 




Truman T. Legg, Senior Systems Analyst 
Alan Slater, Assistant Court Administrator 
Walter F. Germond, Deputy District Attorney 
William J. Morrison, Administrative Services Officer, 

District Attorney's Office 

Don McClure, Manager, Systems and Programming, Computer Sciences 
Corporation 

Keith L. Concannon, Director, Orange County Criminal Justice Council 
Ross F. Penne, Center Director, Computer Sciences Corporation 
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San Bernardino (San Bernardino County), California 

James M. Cramer, District Attorney 

Rex Victor, Assistant District Attorney 

Kay Skawienski, Office of the Public Defender 

Debra A. Haskins, Data Processing Coordinator, San Bernardino County 
Clerk 

Jesse Pointer, Data Processing Coordinator, San Bernardino Municipal 
Courts 

Thomas H. Hudson, Manager, Automated Court Information System 

James R. Johnson, Vice President, Application Development Systems, Inc. 




Robert Webb, Assistant Prosecutor 
Jim Hagen, System Manager 

Ray Rule, Senior Management Analyst, County Executive Office 
Joel Berger, Adult Probation 
Alice Wheatly, Municipal Court 
Pete Kiefer, Superior Court 



Golden, Colorado 

Nolan L. Brown, District Attorney 

Dan B. Fahrney, Chief Deputy District Attorney, District Courts 

Maurice H. Bennett, Jr., Administrator 

Kristen M. Beauchamp, Data Entry Technician 

Mary L. Simon, Data Entry Technician 

Larry Webster, PROMIS Supervisor 

Jerry L. Jorgenson, Chief Deputy District Attorney, Intake and Screening 
Ray Kechter, Chief Screening Officer 
Steve Siegal, Director, Pretrial Services 

C. Stephen Cantrell, Deputy District Attorney, Preliminary Hearings 
Judi Webb, Paralegal, Preliminary Hearings 
Pat Blackard, Clerk, Preliminary Hearings 

Diane Edes, Systems Manager, Regional PROMIS, Colorado District 
Attorney's Council 

Deyrol E. Anderson, Deputy Director, Colorado District Attorney's Council 
Roger H. Allott, Chairman, Regional PROMIS Board of Directors and Chief 
Deputy District Attorney, 18th Judicial Circuit 
James Opp, Director, Jefferson County Department of Data Processing 
Don Haakinson, PROMIS Team Leader, Jefferson County Department of Data 
Processing ' 

Lt. Bruce Glasscock, Detective Division, Lakewood Police Department 
Honorable Anthony F. Vollock, Judge, First Judicial Circuit , 

Honorable Daniel J. Shannon, Presiding Judge, First Judicial Circuit 
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District of Columbia 



Terry Russell, Special Assistant 

David Hetzel, Chief, Misdemeanors 

H. Greene, Chief, Superior Court Division 

Joe Valder, Deputy Chief, Grand Jury Unit 

John Hume, Chief, Felony Trials 

John DePaolis, Assistant U.S. Attorney 

Gloria Dellavalle, Chief, Systems 

John Middleton, Systems, D.C. Office of CJ Analysis 

Sue Ellen Hais, Systems, U.S. Attorney's Office 



Miami (Dade County), Florida 

Henry N. Adorno, Prosecution Management 

Jay Kolosky, Assistant Prosecutor 

Steven Levenson, Administrative Assistant 

°r Services and Information Systems 

Biirstoiloff Gustlce Council 

Dill btoiloff, Clerk’s Office, Dade County Court 

Waukeeqan (Lake County), Illinois 

'^^Services^^'^^’ Administrator, Lake County Department of Management 

Richard Hilton, Department of Management Services 
Rhonda Brandhorst, Department of Management Services 

menrServLs’’”'^''''' Systems, Department of Manage- 

P. Randall Knowles, Assistant State’s Attorney 
Honorable Harry D. Strouse, Judge, 19th Judicial Court 
Lt. Eugene McGaughey, Lake County Sheriff's Department 



Indianapolis (Marion County), Indiana 

Stephen Goldsmith, Marion County Prosecuting Attorney 

Beth Walpole, PROMIS Coordinator, Office of the Prosecuting Attorney 

Bill Divine, Deputy Prosecuting Attorney 

O'Connor, Marion County Data Processing 
^*Attorney*^^ Wieting, Business Manager, Office of the Prosecuting 
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Paul Richwelvky, First Assistant 
Richard Cooper, Assistant Prosecutor 
William Chiquelin, CATCH Project Manager 



New Orleans (Orleans Parish). Louisiana 

Ralph Capatelli, First Deputy 

John Meyer , Assistant Prosecutor 

Robert Early, Assistant Prosecutor 

Lance Afrik, Assistant Prosecutor 

Denis Waldron, Assistant Prosecutor 

Emmett Fremaux, Chief Deputy Clerk, District Court 

Glen Christina, System Manager 

Jim Rousselle, Assistant System Manager 




Barbara Daly, Office of the State's Attorney 
Maryann Willin, Deputy State's Attorney 

Baltimo?f Assignment Commissioner, Supreme Bench of 
Linda Crowley, State Judicial Information Systems 




Executive Assistant to District Attorney 

r F? Assistant to District Attorney 

Jim Caffrey, Assistant Prosecutor ^ 

Bob Long, Assistant Prosecutor 

Joh^^DuffetT’q^T''® Supervisor, Assignments Section 

Mov S Systems Manager, Assignments Section 

Mary McCarthy, Data Recorder/Coder 

Bernie Dwyer, Assistant Prosecutor 

Bob Powers, Assistant Prosecutor 

Jim Lynch, Assistant Prosecutor 

Daniel C. Mullane, Assistant Prosecutor 

Marion Walsh, Legal Administrative Assistant 

Bab^MlJchen""judloni’ Pdllcg Department 

M«d1laL“countJ Dourt, 

®°MldSSx Co“2ntj"^ Information System, Superior Court, 
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Dr. Glen Wallace, Director, Statistical Analysis Center (SAC), 

Oklahoma Crime Commission 
Jim Wilson, ADRS System Manager, SAC 
Jon Steen, Data Analyst, SAC 
Neal Gilson, ADRS Training Officer, SAC 
J. Patrick Sweeney, Systems Analyst, SAC 

Del Woodruff Oklahoma County District Attorney's Office, System Manager 
Jane Bluejacket, Programmer ^ 

Paul D. Boyd, Chief, Identification Section, Oklahoma State Bureau of 
Investigation 

Attorney's Training Coordination Council 
Niles Jackson, Attorney, District Attorney's Training Coordination 
Council 

^°ShIwner°OK Assistant District Attorney, Pottawatomi County, 
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Jack Pessia, PROMIS Coordinator 
Chrys A. Martin, Staff Assistant 
Suzanne Lewis, Data Input Control Clerk 
Dorthea Anderson 
Kelly Bacon, Executive Assistant 
Wayne C. Pearson 

Jack Wilson, Systems Analyst, v Data Processing Authority 
Bob Davidson, Financial Administrator, Data Processing Authority 
Charles Benard, Criminal Coordinator, Circuit Court 
Adele Goggins, System Specialist, Circuit Court' 



Fort Worth (Tarrant County). Texas. 



Wayne Hyde, System Manager 

J. J. Heinemann, Assistant Prosecutor 

Steve Chaney, Assistant Prosecutor 
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Tommy Miller, Assistant Prosecutor 

Assistant, Commonwealth's Attorney's Office 
J.W. Nixon, Data Processing Manager, General Services 
A.C. Hooper, Clerk of Court's Office, District Court 
Charlie Greene, Clerk of Court's Office, Circuit Court 
Bill Garbee, Systems Analyst, Data Processing Division 
Sgt. D.H. Mason, Central Files Division, Norfolk Police Department 
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Norfolk, Virginia (Continued) 

Capt. Niel Koch, Commander. Central n- • • 

Department Central Files Division, Norfolk Police 

'^i^ector of General Services 
Lbs Attorney’s Office 

Lbs a™ Dillsr, Systs^s Analyst, Data^ Processing Division 
Milwaukee (Mil waukee County). 

'■°oiinurt1‘"’ Infor^tlon Systems Coordinator, Clerk 

Honorable William Gardner, Judge 

Donald^Tho^gaard^^Chie^^^ Sheriff's Office 

Robert Erdman, Calendar Clerk^’Glerk^rr^°T!' Division 
Franklin Lott^r, Superintendent Ml Division 
Herman B. John, Deputy District ’At t^r^dy^^ " ^ Corrections 



